Electronic device providing text-related image and method for operating the same

ABSTRACT

An electronic device comprises an input device comprising input circuitry, a display device, a communication circuit, and at least one processor configured to control the electronic device to: receive a text through the input device, transmit first information about the text to a server, control the communication circuit to receive second information associated with an image identified based on an emotional state of a first user, the emotional state of the first user being identified as a result of analysis of the text by a learning model trained using a database for a plurality of texts and a plurality of types of emotion and degrees of emotion and an emotional state of a second user conversing with the first user, and display the image based on the second information associated with the image.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims priority under 35 U.S.C. § 119to Korean Patent Application No. 10-2018-0007955, filed on Jan. 22,2018, in the Korean Intellectual Property Office, the disclosure ofwhich is incorporated by reference herein in its entirety.

BACKGROUND Field

The disclosure relates to electronic devices to provide an image relatedto a received text and methods for operating the electronic devices and,for example, to select or edit an image corresponding to content of thetext and provide the selected or edited image and methods for operatingthe electronic devices.

The disclosure also relates to artificial intelligence (AI) systemsusing machine learning algorithms and their applications.

Description of Related Art

With smartphones or tablet PCs recently in wide use, users maycommunicate messages with others while on the move. Various applicationsare now available for the purpose of message communication. Suchapplications not only communicate text messages but also provide varioususer-friendly functions. For example, an application may send out animage, e.g., emoticon, along with a text, and the user may control theelectronic device to choose and send out her desired image to anotherelectronic device.

Human intelligence-class artificial intelligence (AI) systems are beingutilized in various industry sectors. AI systems learn on their own andget smarter unlike existing rule-based smart systems. The more used, themore precisely AI systems may perceive and understand users' preference.Thus, legacy rule-based smart systems are being gradually replaced withdeep learning-based AI systems.

AI technology may include machine learning (e.g., deep learning) andmachine learning-based component technology.

Machine learning may refer, for example, to an algorithm technique thatitself may classify and learn the features of input data. The componenttechnology may refer, for example, to a technique for simulating thehuman brain's perception and decision capabilities using a machinelearning algorithm (e.g., deep learning), and this may be divided intoseveral technical fields, such as linguistic understanding, visualunderstanding, inference/prediction, knowledge expression, and operationcontrol.

The following are examples of AI applications. Linguistic understandingmay refer, for example, to technology for recognizing andapplying/processing a human being's language or text, and thisencompasses natural language processing, machine translation, dialogsystem, answering inquiries, and speech recognition/synthesis. Visualunderstanding may refer, for example, to a technique of perceiving andprocessing things as do human eyes, and this encompasses objectrecognition, object tracing, image search, human recognition, scenerecognition, space understanding, and image enhancement. Inferenceprediction may refer, for example, to a technique of determining andlogically inferring and predicting information, encompassingknowledge/probability-based inference, optimization prediction,preference-based planning, and recommendation. Knowledge expression mayrefer, for example, to a technique of automatically processing humanexperience information, covering knowledge buildup (dataproduction/classification) and knowledge management (data utilization).Operation control may refer, for example, to a technique of controllingthe motion of robots and driverless car driving, and this encompassesmovement control (navigation, collision, driving) and maneuveringcontrol (behavior control).

To express the user's various emotional states, the electronic deviceprovides multiple candidate images. To find the user's desired one amongthe candidate images, a search needs to be done on each candidate image.Thus, it may take a long time to provide the desired image. There is notechnology related to choosing or editing images corresponding toemotions among a plurality of users. Nor is there technology to mergeimages corresponding to a plurality of users into a single image andprovide the same.

The above information is presented as background information only toassist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

According to various embodiments, an electronic device and method foroperating the electronic device may provide an image selected and/oredited depending on the type and/or degree of the user's emotion.According to various embodiments, an electronic device and method foroperating the electronic device may select and/or edit an imagecorresponding to the emotion among a plurality of users and provide theselected and/or edited image. According to various embodiments, anelectronic device and method for operating the electronic device maymerge images corresponding to a plurality of users into a single imageand provided the merged image.

According to an embodiment, an electronic device comprises an inputdevice comprising input circuitry, a display device, a communicationcircuit, and a processor configured to control the electronic device to:receive a text through the input device, transmit information about thetext through the communication circuit to a server, receive informationassociated with an image identified based on a first user's emotionalstate identified as a result of analysis of the text and an emotionalstate of a second user conversing with the first user, and display theimage through the display device based on the information associatedwith the image.

According to an embodiment, a method for operating an electronic devicecomprises receiving a text, transmitting information about the text to aserver, receiving information associated with an image identified basedon a first user's emotional state identified as a result of analysis ofthe text and an emotional state of a second user conversing with thefirst user, and displaying the image based on the information associatedwith the image.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses exemplary embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

to the above and/or other aspects, features and attendant advantages ofvarious example embodiments of the present disclosure will be moreapparent and readily appreciated from the following detaileddescription, taken in conjunction with the accompanying drawings, inwhich like reference numerals refer to like elements, and wherein:

FIG. 1A is a diagram illustrating an example electronic device, anexample external electronic device, and a server according to anembodiment;

FIG. 1B is a block diagram illustrating an example electronic device, anexample external electronic device, and a server according to anembodiment;

FIG. 2A is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 2B is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 2C is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 2D is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 3 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 4A is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 4B is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 4C is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 5 is a diagram illustrating an example of changing images dependingon a variation in the degree of emotion according to an embodiment;

FIG. 6 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 7A is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 7B is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 7C is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 8A is diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 8B is diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 8C is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 9 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 10A is a diagram illustrating an example of editing an imageaccording to an embodiment;

FIG. 10B is a diagram illustrating an example of editing an imageaccording to an embodiment;

FIG. 10C is a diagram illustrating an example of editing an imageaccording to an embodiment;

FIG. 10D is a diagram illustrating an example of editing an imageaccording to an embodiment;

FIG. 10E is a diagram illustrating an example of editing an imageaccording to an embodiment;

FIG. 11 is a diagram illustrating example screens displayed on anelectronic device according to an embodiment;

FIG. 12A is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 12B is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 13A is a diagram illustrating example screens displayed on anelectronic device according to an embodiment;

FIG. 13B is a diagram illustrating example screens displayed on anelectronic device according to an embodiment;

FIG. 14A is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 14B is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 14C is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 15 is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 16 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 17A is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 17B is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 18A is a diagram illustrating an example process for displaying animage according to an embodiment;

FIG. 18B is a diagram illustrating an example process for displaying animage according to an embodiment;

FIG. 19 is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 20 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 21 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 22A is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 22B is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 23 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 24 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 25 is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 26 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 27 is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 28 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 29 is a diagram illustrating an example screen displayed on anelectronic device and a captured image according to an embodiment;

FIG. 30 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 31A is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 31B is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 32A is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 32B is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 32C is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 33 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 34 is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 35 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment;

FIG. 36 is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment;

FIG. 37A is a diagram illustrating example screens displayed on anelectronic device according to an embodiment;

FIG. 37B is a diagram illustrating example screens displayed on anelectronic device according to an embodiment;

FIG. 38 is a block diagram illustrating an example processor accordingto an embodiment;

FIG. 39 is a block diagram illustrating an example data learning unitaccording to an embodiment; and

FIG. 40 is a block diagram illustrating an example data recognizing unitaccording to an embodiment.

Throughout the drawings, like reference numerals will be understood torefer to like parts, components, and structures.

DETAILED DESCRIPTION

FIG. 1A is a diagram illustrating an example electronic device, anexample external electronic device, and a server according to anembodiment. FIG. 1B is a block diagram illustrating an exampleelectronic device, an example external electronic device, and a serveraccording to an embodiment.

According to an embodiment, an electronic device 101 may communicatewith a server 108, and an external electronic device 102 may communicatewith the server 108. The electronic device 101 may communicate with theexternal electronic device 102 and/or the server 108 through, forexample, and without limitation, a long-range wireless communicationnetwork, or the like. Although not shown, the electronic device 101 maydirectly communicate with the external electronic device 102 through,for example, and without limitation, a short-range wirelesscommunication network, or the like.

According to an embodiment, the electronic device 101 may transmit datato the external electronic device 102 through the server 108 and/orreceive data from the external electronic device 102 through the server108. For example, the electronic device 101 may transmit at least one ofinformation about a text or information about an image to the externalelectronic device 102 through the server 108. The electronic device 101may receive at least one of information about a text or informationabout an image from the external electronic device 102 through theserver 108. In the disclosure, the “electronic device 101, the externalelectronic device 102, and/or the server 108 each may perform aparticular operation” may be appreciated as a processor (e.g., includingprocessing circuitry) 120, a processor (e.g., including processingcircuitry) 121, and/or a processor (e.g., including processingcircuitry) 122 each of which may perform a particular operation. The“electronic device 101, the external electronic device 102, or theserver 108 each may perform a particular operation” may, for example, beappreciated as the processor 120, the processor 121, and/or theprocessor 122 each controlling various elements, such as, for example,and without limitation, hardware, or the like, inside or outside theelectronic device 101, the external electronic device 102, and/or theserver 108 to perform the particular operation. The “electronic device101, the external electronic device 102, or the server 108 each mayperform a particular operation” may be appreciated as a memory 130, amemory 131, and/or a memory 132 each storing instructions which, whenexecuted by at least one of the processor 120, the processor 121, and/orthe processor 122 and/or the hardware, may control an element (e.g.,electronic device, server, or the like) to perform the particularoperation.

According to an embodiment, the electronic device 101 may include aprocessor (e.g., including processing circuitry) 120, a memory 130, aninput device (e.g., including input circuitry) 150, a display device160, and a communication circuit 190. The external electronic device 102may include a processor (e.g., including processing circuitry) 121, amemory 131, an input device (e.g., including input circuitry) 151, adisplay device 161, and a communication circuit 191. The server 108 mayinclude a processor (e.g., including processing circuitry) 122, a memory132, and a communication circuit 192.

According to an embodiment, the processor 120 may include variousprocessing circuitry and execute, e.g., software (e.g., a program) tocontrol at least one other component (e.g., a hardware and/or softwarecomponent) of the electronic device 101 connected with the processor 120and may process and/or compute various data. According to oneembodiment, as at least part of the data processing and/or computation,the processor 120 may load a command and/or data received from anothercomponent (e.g., the communication circuit_190 and/or a sensor module(not shown)) onto a volatile memory, process the command and/or the datastored in the volatile memory, and store resulting data in, for example,a non-volatile memory. According to an embodiment, the processor 120 mayinclude various processing circuitry, such as, for example, and withoutlimitation, a main processor (e.g., a central processing unit (CPU), anapplication processor (AP), or the like), or the like, and an auxiliaryprocessor (e.g., a graphics processing unit (GPU), an image signalprocessor (ISP), a sensor hub processor, a communication processor (CP),or the like), or the like that is operable independently from, and/or inconjunction with, the main processor. The auxiliary processor may beadapted to consume less power than the main processor, or to be specificto a specified function. The auxiliary processor may be implemented asseparate from, or as part of the main processor. The auxiliary processormay control at least some of functions or states related to at least one(e.g., the display device 160 and/or the communication circuit 190) ofthe components of the electronic device 101, instead of the mainprocessor while the main processor is in an inactive (e.g., sleep) stateor along with the main processor while the main processor is an activestate (e.g., executing an application). According to an embodiment, theauxiliary processor (e.g., an image signal processor or a communicationprocessor) may be implemented as part of another component (e.g., thecommunication circuit 190) functionally related to the auxiliaryprocessor.

The memory 130 may store various data used by at least one component(e.g., the processor 120) of the electronic device 101. The various datamay include, for example, and without limitation, software (e.g., theprogram), or the like, and input data, output data, or the like for acommand related thereto. The memory 130 may include a volatile memoryand/or a non-volatile memory.

The input device 150 may include various input circuitry and receive acommand and/or data to be used by other component (e.g., the processor120) of the electronic device 101, from the outside (e.g., a user) ofthe electronic device 101. The input device 150 may include variousinput circuitry, such as, for example, and without limitation, amicrophone, a mouse, a keyboard, or the like.

The display device 160 may visually provide information to the outside(e.g., a user) of the electronic device 101. The display device 160 mayinclude, for example, and without limitation, a display, a hologramdevice, a projector, or the like and control circuitry to control acorresponding one of the display, hologram device, and/or projector.According to an embodiment, the display device 160 may include touchcircuitry adapted to detect a touch, and/or sensor circuitry (e.g., apressure sensor) adapted to measure the intensity of force incurred bythe touch. The input device 150 and the display device 160 may beimplemented, for example, and without limitation, as a touchscreendevice.

The communication circuit 190 may support establishing a direct (e.g.,wired) communication channel or wireless communication channel betweenthe electronic device 101 and an external electronic device (e.g., theexternal electronic device 102 or the server 108) and performingcommunication through the established communication channel. Thecommunication circuit 190 may include one or more communicationprocessors that are operable independently from the processor 120 (e.g.,the application processor (AP)) and supports a direct (e.g., wired)communication or a wireless communication. According to an embodiment,the communication circuit 190 may include a wireless communicationmodule (e.g., a cellular communication module, a short-range wirelesscommunication module, and/or a global navigation satellite system (GNSS)communication module) and/or a wired communication module (e.g., a localarea network (LAN) communication module and/or a power linecommunication (PLC) module). A corresponding one of these communicationmodules may communicate with the external electronic device via a firstnetwork (e.g., a short-range communication network, such as Bluetooth™,wireless-fidelity (Wi-Fi) direct, or infrared data association (IrDA))or a second network (e.g., a long-range communication network, such as acellular network, the Internet, or a computer network (e.g., LAN or widearea network (WAN)). These various types of communication modules may beimplemented using various communication circuitry implemented, forexample, and without limitation, as a single component (e.g., a singlechip), as multi components (e.g., multi chips) separate from each other,or the like. The wireless communication module may identify andauthenticate the electronic device 101 in a communication network, suchas the first network or the second network, using subscriber information(e.g., international mobile subscriber identity (IMSI)) stored in thesubscriber identification module.

The processor 121 and the processor 122 may include substantially thesame components as the processor 120 and/or provide substantially thesame functions as the processor 120. The memory 131 and the memory 132may include substantially the same components as the memory 130 and/orprovide substantially the same functions as the memory 130. Thecommunication circuit 191 and the communication circuit 192 may includesubstantially the same components as the communication circuit 190and/or provide substantially the same functions as the communicationcircuit 190. The input device 151 may include substantially the samecomponents as the input device 150 and/or provide substantially the samefunctions as the input device 150. The display device 161 may includesubstantially the same components as the display device 160 and/orprovide substantially the same functions as the display device 160.

Some of the above-described components may be connected together throughan inter-peripheral communication scheme (e.g., a bus, general purposeinput/output (GPIO), serial peripheral interface (SPI), mobile industryprocessor interface (MIPI), or the like), communicating signals (e.g.,instructions and/or data) therebetween.

According to an embodiment, commands and/or data may be transmitted orreceived between the electronic device 101 and the external electronicdevice 102 via the server 108. The external electronic device 102 may bethe same or different in type from the electronic device 101. Accordingto an embodiment, all or some of operations executed on the electronicdevice 101 may be run on one or more other external electronic devices.According to an embodiment, when the electronic device 101 shouldperform a certain function or service automatically or at a request, theelectronic device 101, instead of, or in addition to, executing thefunction or service on its own, may request an external electronicdevice to perform at least some functions associated therewith. Theexternal electronic device receiving the request may execute therequested functions or additional functions and transfer a result of theexecution to the electronic device 101. The electronic device 101 mayprovide a requested function or service by processing the receivedresult as it is or additionally. To that end, a cloud computing,distributed computing, or client-server computing technique may be used,for example.

FIG. 2A is a flowchart illustrating an example method of operating anelectronic device according to an embodiment. FIG. 2B is a flowchartillustrating an example method of operating an electronic deviceaccording to an embodiment. FIG. 2C is a flowchart illustrating anexample method of operating an electronic device according to anembodiment. FIG. 2D is a flowchart illustrating an example method ofoperating an electronic device according to an embodiment.

According to an embodiment, an electronic device 101 may receive a textin operation 201. For example, the electronic device 101 may store andexecute an application for message communication and provide anexecution screen through the display device 160. The electronic device101 may display a user interface that includes, for example, a softinput panel (SIP) to receive text and an input window to display thereceived text. Based on a touch on at least one object of the SIP, theelectronic device 101 may display a text, or a text combination,corresponding to the touched object, based on the input window.

The electronic device 101 may transmit the text to a server 108 inoperation 203. The electronic device 101 may transmit the text, e.g., atthe user's request. Even without the user's request, the electronicdevice 101 may transmit the text to the server 108 when a new text isentered. The electronic device 101 may transmit the new text alone orthe whole text including the new text to the server 108.

According to an embodiment, the server 108 may analyze the received textand identify the type and degree of the user's emotion corresponding tothe text in operation 205. The server 108 may store an algorithm capableof identifying the type and degree of the user's emotion from the text.For example, the server 108 may store a learning model learned using adatabase for a plurality of texts and a plurality of emotion types anddegrees. According to an embodiment, the server 108 may separate thetext into morphemes and perform keyword analysis on the morphemes. Forexample, the server 108 may separate one sentence per word class andidentify the type and degree of the user's emotion based on the textcorresponding to the noun, adverb, or verb. For example, the server 108may identify that the type of the user's emotion corresponding to afirst text is “sad”. The server 108 may identify the degree of theuser's emotion corresponding to the first text as one of a plurality oflevels. The server 108 may store an algorithm capable of identifying thelevel indicating the user's emotion based on the result of text analysisand identify the degree of emotion using the algorithm. The server 108may identify the type of an action based on a text corresponding to averb. It would be apparent to a skilled artisan that no limitations areposed to schemes for identifying the type and degree of the user'semotion from the text. A configuration for the server 108 to identifythe type and degree of emotion through learning is described in greaterdetail below.

In operation 207, the server 108 may transmit information about theidentified type and degree of emotion to the electronic device 101. Inoperation 209, the electronic device 101 may select an imagecorresponding to the text among a plurality of images or edit anddisplay a raw image based on the type and degree of emotion. Forexample, the image may be an emoticon, and may also be referred to as agraphic object. For example, the electronic device 101 may store aplurality of candidate images. The electronic device 101 may classifyand store the plurality of candidate images per, e.g., type of emotion.Table 1 illustrates an example in which the electronic device 101classifies and stores images according to an embodiment.

TABLE 1 Type of Character emotion Degree of emotion Image firstcharacter sad first level first image second level second image thirdlevel third image happy first level fourth image second level fifthimage third level sixth image second character sad first level seventhimage second level eighth image third level ninth image happy firstlevel tenth image second level eleventh image third level twelfth image

As set forth above, the electronic device 101 may store, per character,at least one image (e.g., the first image to the sixth image)corresponding to the same character. Despite the same type of emotion,the electronic device 101 may store different images based on the degreeof emotion. For example, the electronic device 101 may receiveinformation indicating that the type of emotion corresponding to thefirst text is “happy” and the degree of emotion is the second level fromthe server 108. The electronic device 101 may identify (determine) thata designated character is the first character and provide the fifthimage based thereupon. Where no character is designated, the electronicdevice 101 may identify an image corresponding to a default character.For example, the characters may be images looking like specific animalsor humans or may be implemented in various forms, such as things orgeometric shapes, but it would be apparent to one of ordinary skill inthe art that embodiments of the disclosure are not limited thereto.

Meanwhile, the type of emotion identified by the server 108 may differfrom the type of emotion defined by the electronic device 101. Theelectronic device 101 may select a type of emotion similar to the typeof emotion identified by the server 108. For example, where the type ofemotion identified by the server 108 is “delightful”, the electronicdevice 101 may select a similar type of emotion, “happy”, to the type ofemotion identified by the server 108. The electronic device 101 maydetermine the similarity based on semantics and determine the type ofemotion. Meanwhile, the degree of emotion identified by the server 108may differ from the degree of emotion defined by the electronic device101. For example, the electronic device 101 may select a degree ofemotion similar to the degree of emotion identified by the server 108.For example, where the degree of emotion identified by the server 108 is“high”, the electronic device 101 may select a similar emotion level,“3”, to the degree of emotion identified by the server 108. Theelectronic device 101 may determine the degree based on semantics anddetermine the type of emotion.

The electronic device 101 may select an image corresponding to thecharacter based on the information received from the server 108 amongthe plurality of candidate images and provide the selected image.

According to an embodiment, rather than storing the plurality of images,the electronic device 101 may edit and provide a raw image correspondingto the character based on the information received from the server 108.In this case, the electronic device 101 may store raw images as shown inTable 2.

TABLE 2 Character Raw image first character first raw image secondcharacter second raw image

The electronic device 101 may edit and provide the raw image based onthe type and degree of emotion received from the server 108. Forexample, the electronic device 101 may store edit information as shownin Table 3 and edit the raw image based on the edit information.

TABLE 3 Type of emotion Degree of emotion Edit information sad firstlevel first edit information second level second edit information thirdlevel third edit information happy first level fourth edit informationsecond level fifth edit information third level sixth edit information

For example, the edit information may be information for adjusting, forexample, and without limitation, one or more of the position, shape,color, changing speed of at least one object, or the like, which isdescribed in greater detail below. For example, where the designatedcharacter is identified (determined) to be the first character, theelectronic device 101 may load the first raw image. Corresponding to thetype of emotion being “happy” and the degree of emotion being the “thirdlevel” based on the information received from the server 108, theelectronic device 101 may select the sixth edit information. The sixthedit information may, for example, be edit information for moving up theposition of both ends of the object corresponding to, e.g., “mouth”.Based thereupon, the electronic device 101 may perform an edit formoving up the position of both ends of the object corresponding to the“mouth” of the first raw image and provide an edited image.

According to an embodiment, the electronic device 101 may display theselected or edited image alone or together with the text. The electronicdevice 101 may dynamically vary the image corresponding to the textdisplayed on the input window. For example, where the input windowdisplays “Ah˜”, the electronic device 101 may provide an imagecorresponding to the type of emotion, “disappointed”, and the degree ofemotion, “third level”, based on the information from the server 108.Thereafter, where an additional text is entered so that the input windowdisplays “Ah˜ great˜!!”, the electronic device 101 may replace with anddisplay an image corresponding to the type of emotion, “happy”, and thedegree of emotion, “third level”, based on the information from theserver 108.

In operation 211, the electronic device 101 may transmit the text andthe image to the server 108. For example, corresponding to obtaining atext and image transmit command, the electronic device 101 may transmitthe text and the provided image to the server 108. In operation 213, theserver 108 may transmit the text and image to the external electronicdevice 102. In operation 215, the external electronic device 102 maydisplay the received text and image. Thus, the external electronicdevice 102 may display the image automatically provided based on thetype and degree of the user's emotion from the electronic device 101.

FIG. 2B is a flowchart illustrating an example method of operating anelectronic device according to an embodiment. Where at least someoperations of FIGS. 2B, 2C and 2D are substantially the same or similarto at least some operations of FIG. 2A, the operations are describedbriefly.

According to an embodiment, the electronic device 101 may receive a textin operation 221. In operation 223, the electronic device 101 maytransmit the text to the server 108. In operation 225, the server 108may analyze the text and identify the type and degree of the user'semotion corresponding to the text. In operation 227, the server 108 mayselect an image based on the type of emotion and the degree of emotion.Unlike in FIG. 2A, the server 108 may store a plurality of candidateimages and/or a raw image for editing, and the server 108 may select animage corresponding to the type and degree of the user's emotionidentified. In this case, the server 108 may identify a charactercorresponding to the user's account or use a default character. Inoperation 229, the server 108 may transmit the selected image to theelectronic device 101. The server 108 may also transmit an edited image.In operation 231, the electronic device 101 may receive a request fortransmitting the text and the selected image to the external electronicdevice 102. For example, the electronic device 101 may detect adesignation of a “transmit” button on the user interface. In operation233, the electronic device 101 may transmit the text and the selectedimage to the server 108. In operation 235, the server 108 may transmitthe text and the selected image to the external electronic device 102.In operation 237, the external electronic device 102 may display thetext and the selected image.

Referring to FIG. 2C, the electronic device 101 may receive a text inoperation 221. In operation 223, the electronic device 101 may transmitthe text to the server 108. In operation 225, the server 108 may analyzethe text and identify the type and degree of the user's emotioncorresponding to the text. In operation 227, the server 108 may selectan image based on the type of emotion and the degree of emotion. Inoperation 230, the server 108 may transmit information associated withthe selected image. In other words, rather than transmitting the imageitself to the electronic device 101, the server 108 may transmitinformation for identifying the image or edit information about theimage to the electronic device 101. In operation 232, the electronicdevice 101 may select or edit the image based on the receivedinformation associated with the image. The electronic device 101 mayselect an image among a plurality of candidate images using theinformation for identifying an image. The electronic device 101 mayapply the edit information to the raw image, producing an edited image.In operation 233, the electronic device 101 may transmit the text andthe selected image to the server 108. In operation 235, the server 108may transmit the text and the selected image to the external electronicdevice 102. In operation 237, the external electronic device 102 maydisplay the text and the selected image.

FIG. 2D is a flowchart illustrating an example method of operating anelectronic device according to an embodiment.

In operation 241, according to an embodiment, the electronic device 101may receive a text. In operation 243, the electronic device 101 mayanalyze the text and identify the type and degree of the user's emotioncorresponding to the text. In other words, the type and degree of theuser's emotion may be identified not by the server 108 but by theelectronic device 101. The electronic device 101 may identify the typeand degree of the user's emotion in substantially the same manner asdoes the server 108 as described above in FIG. 2A. In operation 245, theelectronic device 101 may select or edit the image based on the type anddegree of emotion. In operation 247, the electronic device 101 maytransmit the text and the selected image to the server 108. In operation249, the server 108 may transmit the text and the selected image to theexternal electronic device 102. In operation 251, the externalelectronic device 102 may display the text and the selected image.According to an embodiment, the electronic device 101 may transmit theimage alone through the server 108 to the external electronic device102.

As set forth above, the identifying of the type and degree of the user'semotion may be carried out by the server 108 or the electronic device101, and the selecting or editing of the image may also be performed bythe server 108 or the electronic device 101. Accordingly, theconfiguration in which the electronic device 101 identifies the type anddegree of emotion may be replaced with the configuration in which theserver 108 identifies the type and degree of emotion or vice versa. Theconfiguration in which the electronic device 101 selects or edits theimage based on the type and degree of emotion may be replaced with theconfiguration in which the server 108 selects or edits the image basedon the type and degree of emotion or vice versa.

FIG. 3 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment.

According to an embodiment, an electronic device 101 may receive a textin operation 301. In operation 303, the electronic device 101 maytransmit the text to the server 108. In operation 305, the server 108may analyze the text and identify the type and degree of the user'semotion and the type of action corresponding to the text. As describedabove in connection with FIG. 2A, the server 108 may identify the typeand degree of the user's emotion and may further identify the type ofaction. For example, the server 108 may identify the type of actionbased on a keyword corresponding to the verb in the text. In operation307, the server 108 may transmit information about the type and degreeof emotion and the type of action to the electronic device 101. Inoperation 309, the electronic device 101 may select or edit an imagecorresponding to the text among a plurality of images based on the typeand degree of emotion and the type of action. For example, theelectronic device 101 may include candidate images per type of emotion,degree of emotion, and type of action and select an image from among thecandidate images based on information received from the server 108. Theelectronic device 101 may store edit information per type of action. Theelectronic device 101 may identify the edit information based on theinformation received from the server 108 and may apply the editinformation to a raw image, producing an edited image. In operation 311,the electronic device 101 may transmit the text and the selected imageto the server 108. In operation 313, the server 108 may transmit thetext and the selected image to the external electronic device 102. Inoperation 315, the external electronic device 102 may display the textand the selected image. According to an embodiment, as described abovein connection with FIGS. 2B, 2C and 2D, the electronic device 101 mayanalyze the text and identify the type of action. The server 108 mayalso select or edit the image.

FIG. 4A is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment. FIG. 4B is a diagramillustrating an example displayed on an electronic device according toan embodiment. FIG. 4C is a diagram illustrating an example displayed onan electronic device according to an embodiment.

According to an embodiment, the electronic device 101 may transmit thetext 501 “Hoyoung. Stop it” displayed on the input window as shown inFIG. 4A to the server 108. As a result of analyzing the text “Hoyoung.Stop it”, the server 108 may identify (determine) that the type of theuser's emotion is “unpleasant”, and the degree of emotion is the firstlevel. The server 108 may transmit the identified information, e.g., thetype of emotion, “unpleasant”, and the degree of emotion, the firstlevel, to the electronic device 101. The electronic device 101 maydisplay the first image 511 based on the type of emotion, “unpleasant”,and the degree of emotion, the first level. The electronic device 101may transmit the text 502 “Hoyoung. Please stop it” displayed on theinput window as shown in FIG. 4B to the server 108. For example, upondetecting the user's additional entry of “please” between a portion ofthe text, “Hoyoung”, and another portion of the text, “stop it”, theelectronic device 101 may transmit the added portion, “Please”, or thewhole text, “Hoyoung. Please stop it”, to the server 108. Based on aresult of analyzing the text, the server 108 may identify that the typeof emotion is “unpleasant”, and the degree of emotion is the secondlevel and may transmit the information to the electronic device 101. Theelectronic device 101 may display the second image 512 based on the typeof emotion, “unpleasant”, and the degree of emotion, the second level.

The electronic device 101 may transmit the text 503 “Hoyoung. You needstop it” displayed on the input window as shown in FIG. 4C to the server108. Based on a result of analyzing the text, the server 108 mayidentify that the type of emotion is “embarrassed”, and the degree ofemotion is the first level and may transmit the information to theelectronic device 101. The electronic device 101 may display the thirdimage 513 based on the type of emotion, “embarrassed”, and the degree ofemotion, the first level.

FIG. 5 is a diagram illustrating an example of changing images dependingon a variation in the degree of emotion according to an embodiment.

Referring to FIG. 5, the electronic device 101 may store a first image531 corresponding to a first degree of emotion, a second image 532corresponding to a second degree of emotion, a third image 533corresponding to a third degree of emotion, and a fourth image 534corresponding to a fourth degree of emotion. The electronic device 101may store the first image 531 to the fourth image 534 corresponding tothe type of emotion, “outraged”, of a first character. Thus, where, as aresult of analysis by the server 108 or the electronic device 101, thetype of emotion is identified as “outrageous”, and the character theelectronic device 101 is currently using is the first character or adefault character is the first character, the electronic device 101 mayselect any one among the first image 531 to the fourth image 534depending upon the degree of emotion.

FIG. 6 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment of the disclosure. Theembodiment related to FIG. 6 is described in greater detail withreference to FIGS. 7A, 7B and 7C. FIG. 7A is a diagram illustrating anexample screen displayed on an electronic device according to anembodiment. FIG. 7B is a diagram illustrating an example screendisplayed on an electronic device according to an embodiment. FIG. 7C isa diagram illustrating an example screen displayed on an electronicdevice according to an embodiment.

According to an embodiment, an electronic device 101 may receive a textin operation 601. In operation 603, the electronic device 101 mayanalyze the text and identify the type of the user's emotioncorresponding to the text. For example, the electronic device 101 maydirectly identify the type of the user's emotion or may receiveinformation about the type of emotion identified by the server 108. Inoperation 605, the electronic device 101 may display an imagecorresponding to the identified type of emotion. For example, theelectronic device 101 may display a first image 711 corresponding to thetype of emotion, “unpleasant”, identified corresponding to the text 701“Hoyoung. Stop it!!” as shown in FIG. 7A. The first image 711 may be animage corresponding to a default degree of emotion, e.g., the firstlevel.

In operation 607, the electronic device 101 may receive a degree ofemotion. For example, the electronic device 101 may detect a touch 721on the touchscreen (e.g., a message send button) as shown in FIG. 7A andmay detect a touch's continuous movement 722 (e.g., a drag input, flickinput, or the like) in a first direction as shown in FIG. 7B. Inoperation 609, the electronic device 101 may replace the displayed imagewith a different (e.g., another) image or edit the displayed image basedon the received degree of emotion. For example, as shown in FIG. 7B, theelectronic device 101 may receive the touch's continuous movement 722 inthe first direction as an emotion degree change command and display asecond image 712 corresponding to the degree of emotion variedcorresponding thereto.

According to an embodiment, the electronic device 101 may receive anemotion type change command and, accordingly, vary and display theimage. For example, as shown in FIG. 7C, the electronic device 101 maydetect a touch's continuous movement 723 in a second direction. Theelectronic device 101 may receive the touch's continuous movement 723 inthe second direction as an emotion type change command and display athird image 713 corresponding to the type of emotion variedcorresponding thereto. It will be apparent to one of ordinary skill inthe art that differentiating between the emotion type change command andthe emotion degree change command based on the direction of the touch'scontinuous movement is merely an example and the emotion type changecommand or the emotion degree change command is not limited thereby orthereto as long as it is an input that may be differentiated.

FIG. 8A is diagram illustrating an example screen displayed on anelectronic device according to an embodiment. FIG. 8B is diagramillustrating an example screen displayed on an electronic deviceaccording to an embodiment. FIG. 8C is a diagram illustrating an examplescreen displayed on an electronic device according to an embodiment.

Referring to FIG. 8A, according to an embodiment, the electronic device101 may display a plurality of candidate images 811, 812, 813, 814, 815and 816 corresponding to a first character, along with a text 801. Asshown in FIG. 8B, the electronic device 101 may detect a touch'scontinuous movement, detect the touch being directed toward a secondimage 821 and, accordingly, display the second image 821 to berelatively larger than the other images. Further, the electronic device101 may identify the degree of emotion based on the distance of thetouch's continuous movement. As shown in FIG. 8C, the electronic device101 may detect the touch being directed toward a sixth image 822 and,accordingly, display the sixth image 822 to be relatively larger thanthe other images. Upon detecting a confirm command on the image, theelectronic device 101 may select the image as an image to betransmitted.

FIG. 9 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment of the disclosure. Theembodiment of FIG. 9 is described in greater detail with reference toFIGS. 10A, 10B, 10C, 10D and 10E. FIG. 10A is a diagram illustrating anexample of editing an image according to an embodiment. FIG. 10B is adiagram illustrating an example of editing an image according to anembodiment. FIG. 10C is a diagram illustrating an example of editing animage according to an embodiment. FIG. 10D is a diagram illustrating anexample of editing an image according to an embodiment. FIG. 10E is adiagram illustrating an example of editing an image according to anembodiment.

According to an embodiment, an electronic device 101 may receive a textin operation 901. In operation 903, the electronic device 101 mayanalyze the text and identify the type and degree of the user's emotioncorresponding to the text. The electronic device 101 may receive thetype and degree of the user's emotion corresponding to the text from theserver 108. In operation 905, the electronic device 101 may display animage corresponding to the identified type of emotion. In operation 907,the electronic device 101 may select, for example, and withoutlimitation, one of the position, shape, color, changing speed, or thelike of at least one object in the image corresponding to the degree ofemotion. In other words, the electronic device 101 may edit the image.In operation 909, the electronic device 101 may reflect a selectedproperty to (e.g., in/on) the image and display the image.

For example, as shown in FIG. 10A, the electronic device 101 mayidentify at least one object 1002, 1003, 1004, and 1005 of a raw image1001 of a first character. The electronic device 101 may vary at leastsome of the at least one object 1002, 1003, 1004, and 1005 based on atleast one of the degree of emotion or the type of emotion. For example,the electronic device 101 may perform an edit for moving up both ends ofthe object 1005 corresponding to the “mouth” based on the type ofemotion, “happy”, and, accordingly, provide an edited image 1010reflecting the selected property 1006. For example, the electronicdevice 101 may determine the degree of moving up based on the degree ofemotion, “happy”. For example, based on the type of emotion,“outrageous”, the electronic device 101 may perform an edit for movingdown both ends of the object 1005 to provide a modified object 1024corresponding to the “mouth” and an edit for moving up the left end ofthe object 1002 to provide a modified object 1021 corresponding to the“left eye” and the right end of the object 1003 to provide a modifiedobject 1022 corresponding to the “right eye”. For example, theelectronic device 101 may determine the degree of moving based on thedegree of emotion, “outraged”. Thus, the electronic device 101 mayprovide an edited image 1020.

Referring to FIG. 10B, the electronic device 101 may change the color ofthe object while reshaping the object depending on the degree ofemotion. For example, the electronic device 101 may provide images 1031,1032, and 1033 in different colors depending on the degree of emotionfor the type of emotion, “outraged”.

Referring to FIGS. 10C and 10D, the electronic device 101 may perform anedit in such a manner as to recombine an image resource with the rawimage. The electronic device 101 may store together various resources1042, 1043, 1044, 1045, 1052, 1053, 1054, 1055, and 1056 which may berecombined with the raw images 1041 and 1051. The electronic device 101may select at least some of the resources 1042, 1043, 1044, 1045, 1052,1053, 1054, 1055, and 1056 corresponding to at least one of the type ofemotion or the degree of emotion and perform an edit in such a manner asto recombine the selected resource with the raw image 1041 or 1051.

Referring to FIG. 10E, the electronic device 101 may perform an edit insuch a manner as to adjust the moving speed of the image correspondingto the type of emotion and the degree of emotion. For example, theelectronic device 101 may sequentially display a plurality of images togive an animation effect. The electronic device 101 may adjust the imagechange speed for the animation effect depending on, e.g., the degree ofemotion. For example, the electronic device 101 may play a firstanimation image 1061 at a first speed, a second animation image 1062 ata second speed, a third animation image 1063 at a third speed, and afourth animation image 1064 at a fourth speed.

FIG. 11 is a diagram illustrating example screens displayed on anelectronic device according to an embodiment. Referring to FIG. 11, theelectronic device 101 may display a text 1101 and an image 1102corresponding to the text 1101. According to various exampleimplementations, the electronic device 101 may display the text 1104 ina word balloon for the image 1103 as shown on the right of FIG. 11. Theelectronic device 101 may, for example, determine the shape of the wordballoon depending on the type of emotion.

FIG. 12A is a flowchart illustrating an example method of operating anelectronic device according to an embodiment. FIG. 12B is a flowchartillustrating an example for operating an electronic device according toan embodiment. The embodiment of FIGS. 12A and 12B is described ingreater detail with reference to FIGS. 13A and 13B. FIG. 13A is adiagram illustrating example screens displayed on an electronic deviceaccording to an embodiment. FIG. 13B is a diagram illustrating examplescreens displayed on an electronic device according to an embodiment.

Referring to FIG. 12A, according to an embodiment, the electronic device101 may display a text input window and wait for text entry in operation1201. For example, as shown in FIG. 13A, the electronic device 101 maydisplay the text input window 1301. In operation 1203, the electronicdevice 101 may determine whether a new text is entered. The electronicdevice 101 may also display a text send button 1303. The electronicdevice 101 may display a default image 1311. The electronic device 101may detect entry of a text 1302. In operation 1205, the electronicdevice 101 may identify the type and degree of the user's emotioncorresponding to the whole text reflecting the new text. For example,the electronic device 101 may be triggered by entry of the new text totransmit the text to the server 108 and receive the type and degree ofthe user's emotion from the server 108. In operation 1207, theelectronic device 101 may select or edit the image based on the type anddegree of the user' emotion. Thus, the electronic device 101 may displayan image 1312 corresponding to the text 1302.

Referring to FIG. 12B, according to an embodiment, the electronic device101 may display the text input window and receive a text in operation1211. For example, as shown in FIG. 13B, the electronic device 101 maydisplay the text input window 1301 and receive and display the text1302. In operation 1213, the electronic device 101 may determine whethera text analysis request is obtained. For example, the electronic device101 may determine whether a send button 1303 is designated (1304) and,upon detecting a designation 1304 of the send button 1303, send the text1302 to the server 108 or perform analysis. Upon obtaining a textanalysis request, in operation 1215, the electronic device 101 mayidentify the type and degree of the user's emotion corresponding to thewhole text reflecting the new text. For example, the electronic device101 may be triggered by entry of the new text to transmit the text tothe server 108 and receive the type and degree of the user's emotionfrom the server 108. In operation 1217, the electronic device 101 mayselect or edit the image based on the type and degree of the user'emotion. Thus, the electronic device 101 may display an image 1312corresponding to the text 1302.

FIG. 14A is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment. FIG. 14B is a diagramillustrating an example screen displayed on an electronic deviceaccording to an embodiment. FIG. 14C is a diagram illustrating anexample screen displayed on an electronic device according to anembodiment.

According to an embodiment, the electronic device 101 may display a textinput window 1401, a character selection region 1402, and an SIP region1403 as shown in FIG. 14A. Where any one character is selected from thecharacter selection region 1402 and a text is entered, the electronicdevice 101 may identify the type and degree of the user's emotioncorresponding to the text. As shown in FIG. 14B, the electronic device101 may display an image 1412 based on the type and degree of the user'semotion and corresponding to the character. The electronic device 101may display a text 1411 received in association with the image 1412 anddisplay an enlarge button 1413. Upon detecting a designation of theenlarge button 1413, the electronic device 101 may display an enlargedimage 1421 and text and a shrink button 1422 as shown in FIG. 14C. Upondetecting a designation of the shrink button 1422, the electronic device101 may shrink the image back and display.

FIG. 15 is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment.

According to an embodiment, the electronic device 101 may display a text1501 received from a second user and a text 1502 received from a firstuser. The electronic device 101 may display a text 1503 received throughthe SIP. As described above, the electronic device 101 may display animage 1504 corresponding to the text 1503 in a relatively small size.For example, the electronic device 101 may display the image 1504 inpreview. The electronic device 101 may determine the color of the image1504 based on the type and degree of the user's emotion identified basedon the text 1503.

FIG. 16 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment of the disclosure. Theembodiment of FIG. 16 is described in greater detail with reference toFIGS. 17A and 17B. FIG. 17A is a diagram illustrating an example screendisplayed on an electronic device according to an embodiment. FIG. 17Bis a diagram illustrating an example screen displayed on an electronicdevice according to an embodiment.

According to an embodiment, an electronic device 101 may receive a firsttext in operation 1601. In operation 1603, the electronic device 101 maytransmit the first text to the server 108. In operation 1605, theexternal electronic device 102 may receive a second text. In operation1607, the external electronic device 102 may transmit the second text tothe server 108. The server 108 may transmit the first text to theexternal electronic device 102 and the second text to the electronicdevice 101. Accordingly, as shown in FIG. 17A or 17B, the electronicdevice 101 may transmit or receive a plurality of texts 1701, 1703,1705, and 1709 or a plurality of texts 1731, 1733, 1735, and 1737.

In operation 1609, the electronic device 101 may receive a third text.For example, the electronic device 101 may receive the text 1711 “Youhave a death wish?!” as shown in FIG. 17A. In operation 1611, theelectronic device 101 may transmit the third text 1711 to the server108. In operation 1613, the server 108 may analyze at least part of thefirst text, the second text, and/or the third text and identify the typeand degree of the user's emotion corresponding to the third text. Inother words, the server 108 may identify the type and degree of theuser's emotion based on the prior conversation and the text currentlyentered. For example, the server 108 may store an algorithm capable ofanalyzing the prior conversation and identify the context of the currentsentence. The server 108 may store a learning model learned using adatabase for, e.g., a plurality of conversations and a plurality ofemotion types and degrees. The server 108 may thus identify the type anddegree of the user's emotion corresponding to the current text based onthe prior conversation. For example, as a result of analyzing the text1711 “You have a death wish?!” based on the prior conversation, theserver 108 may identify that the type of the user's emotion is“outraged”, and transmit information thereabout to the electronic device101 as shown in FIG. 17A. The electronic device 101 may display a firstimage 1713 corresponding to the degree of emotion, “outraged”. Forexample, as a result of analyzing the text 1739 “That's so dope!” basedon the prior conversation, the server 108 may identify that the type ofthe user's emotion is “delightful”, and transmit information thereaboutto the electronic device 101 as shown in FIG. 17B. The electronic device101 may display a second image 1741 corresponding to the degree ofemotion, “delightful”. In other words, despite the same texts, theserver 108 (or the electronic device 101) may identify different typesof emotion or different degrees of emotion depending on the priorconversation. In operation 1615, the server 108 may transmit informationabout the type and degree of emotion to the electronic device 101. Inoperation 1617, the electronic device 101 may select or edit an imagecorresponding to the text among a plurality of images based on the typeand degree of emotion. In operation 1619, the electronic device 101 maytransmit the text and the image to the server 108. In operation 1621,the server 108 may transmit the text and the image to the externalelectronic device 102. The external electronic device 102 may displaythe text and the image in operation 1623.

FIG. 18A is a diagram illustrating an example process for displaying animage according to an embodiment. FIG. 18B is a diagram illustrating anexample process for displaying an image according to an embodiment. Asshown in FIG. 18A, the electronic device 101 may display a received text1800 on a text input window and an image 1801 corresponding thereto onthe text input window. The electronic device 101 may display an object1810 indicating a word balloon corresponding to the text and an object1802 corresponding to the image. The electronic device 101 may displaythe object 1811 indicating the word balloon corresponding to the textand the object 1803 corresponding to the image to come further insidethe conversation window over time. As time elapses, the electronicdevice 101 may display the object 1812 corresponding to the word balloonincluding the text and the image 1804. Accordingly, the electronicdevice 101 may provide such an effect as if the text and the image movefrom the text input window to the conversation window. Meanwhile, theelectronic device 101 may display various images on the conversationwindow. For example, since the image corresponding to a first user'saccount may differ depending on the type of the user's emotion for thetext, various images may be mixed inside the conversation window. Asshown in FIG. 18B, the electronic device 101 may move and display theimage 1832 in a first position 1842, a second position 1852, and a thirdposition 1862 over time. The electronic device 101 may move and displaythe text 1831 in a fourth 1841, a fifth position 1851, and a sixthposition 1861.

FIG. 19 is a diagram illustrating example screens displayed on anelectronic device according to an embodiment.

As shown in FIG. 19, the electronic device 101 may display an imagecorresponding to the type and degree of the user's emotion and the typeof action. From a first text 1901 reading “hard..”. the electronicdevice 101 may identify that the type of the user's emotion is“disappointed” and the degree of emotion is the first level. Theelectronic device 101 may not identify the type of action from the firsttext 1901. In this case, the electronic device 101 may display a firstimage 1911 based on the identified type and degree of the user'semotion. From a second text 1902 reading “hard to keep . . . ”. theelectronic device 101 may identify that the type of the user's emotionis “disappointed” and the degree of emotion is the first level and thatthe type of action is crossing and placing both arms on the chest, e.g.,an action to symbolize keeping something. Thus, the electronic device101 may display a second image 1912. The second image 1912 is the sameas the first image 1911 in the facial part associated with the user'semotion and differ from the first image 1911 in the body part associatedwith the action. From a third text 1903 reading “really hard to keep”the electronic device 101 may identify that the type of the user'semotion is “disappointed” and the degree of emotion is the second leveland that the type of action is crossing and placing both arms on thechest, e.g., an action to symbolize keeping something. Thus, theelectronic device 101 may display a third image 1913. The third image1913 is the same as the second image 1912 in the body part associatedwith the action and differ from the second image 1912 in the facial partassociated with the user's emotion. From a fourth text 1904 reading“really hard to keep!!!” the electronic device 101 may identify that thetype of the user's emotion is “disappointed” and the degree of emotionis the third level and that the type of action is crossing and placingboth arms on the chest, e.g., an action to symbolize keeping something.Thus, the electronic device 101 may display a fourth image 1914. Thefourth image 1914 is the same as the third image 1913 in the body partassociated with the action and differ from the third image 1913 in thefacial part associated with the user's emotion.

FIG. 20 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment of the disclosure.

According to an embodiment, an electronic device 101 may receive a textin operation 2001. In operation 2003, the electronic device 101 maytransmit the text to the server 108. In operation 2005, the server 108may analyze the text, identify a plurality of types and degrees of theuser's emotion corresponding to the text, and select one type of emotionor identify a compound emotion based on the plurality of types ofemotion and the degree of each emotion. For example, the server 108 mayidentify a first type of emotion and a first degree of emotion, which isthe degree of emotion fo the first type of emotion and a second type ofemotion and a second degree of emotion, which is the degree of emotionfor the second type of emotion, based on the text. The server 108 mayselect any one of the first type of emotion and the second type ofemotion. For example, the server 108 may manage the priority per type ofemotion and select any one based on the identified priority. The server108 may select any one using a result of analysis of a priorconversation. The server 108 may compare the first degree of emotion andthe second degree of emotion to select the higher level of type ofemotion. According to an embodiment, the server 108 may identify acompound emotion of the first type of emotion and the second type ofemotion. The server 108 may also consider the first degree of emotionand the second degree of emotion upon identifying the compound emotion.

In operation 2007, the server 108 may transmit information about theselected type of emotion or information about the compound emotion tothe electronic device 101. In operation 2009, the electronic device 101may select or edit an image corresponding to the text among theplurality of images based on the information about the selected type ofemotion or the information about the compound emotion. In operation2011, the electronic device 101 may transmit the text and the image tothe server 108. In operation 2013, the server 108 may transmit the textand the image to the external electronic device 102. In operation 2015,the external electronic device 102 may display the text and the image.

FIG. 21 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment of the disclosure.

According to an embodiment, an electronic device 101 may receive a textin operation 2101. In operation 2103, the electronic device 101 maytransmit the text to the server 108. In operation 2105, the server 108may analyze the text, identify a plurality of verbs corresponding to thetext, and select one verb or identify a compound verb corresponding tothe text. The text may include a plurality of verbs, and the server 108may extract the plurality of verbs from the text. The server 108 mayselect any one among the plurality of verbs. For example, the server 108may analyze a prior conversation and, as a result of analysis, select averb more appropriate for the prior conversation. The server 108 mayidentify a compound verb reflecting all of the plurality of verbs. Inoperation 2107, the server 108 may transmit information about theselected verb or information about the compound verb. In operation 2109,the electronic device 101 may select or edit an image corresponding tothe text among the plurality of images based on the information aboutthe selected verb or the information about the compound verb. Inoperation 2111, the server 108 may transmit the text and the image tothe server 108. In operation 2113, the electronic device 101 maytransmit the text and the image to the external electronic device 102.In operation 2115, the external electronic device 102 may display thetext and the image.

FIG. 22A is a flowchart illustrating an example method of operating anelectronic device according to an embodiment of the disclosure.

According to an embodiment, an electronic device 101 may receive a textin operation 2101. In operation 2203, the electronic device 101 maycapture or obtain an image of the user. In operation 2205, theelectronic device 101 may transmit the text and the captured image tothe server 108. In operation 2207, the server 108 may analyze the textand image and identify the type and degree of the user's emotioncorresponding to the text. For example, the server 108 may identify theuser's face from the image and, as a result of analysis of face,identify the user's emotional state. The server 108 may store analgorithm capable of identifying the type and degree of the user'semotion from the image. For example, the server 108 may store a learningmodel learned using a database for a plurality of images and a pluralityof emotion types and degrees. The server 108 may identify the type anddegree of the user's emotion obtained by analyzing the text and the typeand degree of the user's emotion obtained by analyzing the images. Theserver 108 may identify the type and degree of the user's emotion basedon, e.g., the above-described process of compound emotion. According toan embodiment, the server 108 may identify the type and degree of theuser's emotion only using the result of image analysis. In operation2209, the server 108 may transmit information about the type and degreeof emotion to the electronic device 101. In operation 2211, theelectronic device 101 may select or edit an image corresponding to thetext among a plurality of images based on the type and degree ofemotion. In operation 2213, the electronic device 101 may transmit thetext and the image to the server 108. In operation 2215, the server 108may transmit the text and the image to the external electronic device102. In operation 2217, the external electronic device 102 may displaythe text and the image.

FIG. 22B is a flowchart illustrating an example method of operating anelectronic device according to an embodiment of the disclosure.

According to an embodiment, an electronic device 101 may receive a textin operation 2231. In operation 2233, the electronic device 101 mayidentify a user property. For example, the electronic device 101 mayidentify user-related information, such as the user's age, stateinformation as per analysis of the user's biometric information, or theuser's gender. In operation 2235, the electronic device 101 may transmitthe text and the information about user property to the server 108. Inoperation 2237, the server 108 may analyze the text and the userproperty and identify the type and degree of the user's emotioncorresponding to the text. In operation 2239, the server 108 maytransmit information about the type and degree of emotion to theelectronic device 101. In operation 2241, the electronic device 101 mayselect or edit an image corresponding to the text among a plurality ofimages based on the type and degree of emotion. In operation 2243, theelectronic device 101 may transmit the text and the image to the server108. In operation 2245, the server 108 may transmit the text and theimage to the external electronic device 102. In operation 2247, theexternal electronic device 102 may display the text and the image.

FIG. 23 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment of the disclosure.

According to an embodiment, in operation 2301, the electronic device 101may transmit the first text to the server 108. In operation 2303, theexternal electronic device 102 may transmit the second text to theserver 108. In operation 2305, the server 108 may analyze a first user'semotional state, e.g., at least one of the type and degree of emotion,based on the first text and a second user's emotional state, e.g., atleast one of the type and degree of emotion, based on the second text.In operation 2307, the server 108 may identify and transmit informationassociated with a first graphic object corresponding to the first textand the first user based on the first user's emotional state and thesecond user's emotional state. For example, the server 108 may transmitinformation associated with one or more graphic objects corresponding tothe first user and the first text and corresponding to the emotionalstates of both the users to the electronic device 101. For example, theserver 108 may transmit information associated with a single image tothe electronic device 101 or information associated with a compoundimage obtained by synthesizing a plurality of images to the electronicdevice 101. In operation 2309, the server 108 may identify informationassociated with a second graphic object corresponding to the second userand the second text based on the first user's emotional state and thesecond user's emotional state and transmit the information to theexternal electronic device 102. In operation 2311, the electronic device101 may provide a first graphic object. In operation 2313, the externalelectronic device 102 may provide a second graphic object.

For example, the electronic device 101 may transmit a text reading“Hoyoung. Stop it” to the server 108. The server 108 may identify thatthe type of emotion corresponding to the first user of the electronicdevice 101 is “outraged”. Corresponding thereto, the external electronicdevice 102 may transmit the text “Sorry. I'll stop it” to the server108. The server 108 may provide a graphic object corresponding to thetext “Hoyoung. Stop it” from the electronic device 101. For example,based on the type of the first user's emotion being “outraged” and thetype of emotion of the second user of the external electronic device 102being “sorry”, the electronic device 101 may provide the electronicdevice 101 with a graphic object corresponding to a relatively lowdegree of emotion and the type of emotion, “outraged”, corresponding tothe text “Hoyoung. Stop it”. The external electronic device 102 maytransmit the text “What did I do wrong?” to the server 108. Also in thiscase, the server 108 may provide a graphic object corresponding to thetext “Hoyoung. Stop it” from the electronic device 101. For example,based on the type of the first user's emotion being “outraged” and thetype of emotion of the second user of the external electronic device 102being “outraged”, the electronic device 101 may provide the electronicdevice 101 with a graphic object corresponding to a relatively highdegree of emotion and the type of emotion, “outraged”, corresponding tothe text “Hoyoung. Stop it”. As set forth above, the server 108 mayprovide a different graphic object depending on the emotion of the usersattending the conversation for the same text (e.g., “Hoyoung. Stop it”).

For example, the electronic device 101 may transmit a text reading“Hoyoung. Stop it” to the server 108. The server 108 may identify thatthe type of emotion corresponding to the first user of the electronicdevice 101 is “outraged”. Corresponding thereto, the external electronicdevice 102 may transmit the text “Sorry. I'll stop it” to the server108. The server 108 may provide a graphic object corresponding to thetext “Hoyoung. Stop it” from the electronic device 101. For example,based on the type of the first user's emotion being “outraged” and thetype of emotion of the second user of the external electronic device 102being “sorry”, the electronic device 101 may provide the electronicdevice 101 with a graphic object which is a combination of a sub graphicobject corresponding to a relatively low degree of emotion and the typeof emotion, “outraged”, and a sub graphic object corresponding to thetype of the second user's emotion, “sorry”, corresponding to the text“Hoyoung. Stop it”. The external electronic device 102 may transmit thetext “What did I do wrong?” to the server 108. Also in this case, theserver 108 may provide a graphic object corresponding to the text“Hoyoung. Stop it” from the electronic device 101. For example, based onthe type of the first user's emotion being “outraged” and the type ofemotion of the second user of the external electronic device 102 being“outraged”, the electronic device 101 may provide the electronic device101 with a graphic object which is a combination of a sub graphic objectcorresponding to a relatively high degree of emotion and the type ofemotion, “outraged”, and a sub graphic object corresponding to the typeof the second user's emotion, “outraged”, corresponding to the text“Hoyoung. Stop it”. As set forth above, the server 108 may also providea graphic object created by merging sub graphic objects each of whichcorresponds to a respective one of the plurality of users, correspondingto the text from one user.

FIG. 24 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment. The embodiment related toFIG. 24 is described in greater detail with reference to FIG. 25. FIG.25 is a diagram illustrating an example displayed on an electronicdevice according to an embodiment.

According to an embodiment, in operation 2401, the electronic device 101may transmit information about a first graphic object corresponding to afirst user and a first text to the server 108. For example, theelectronic device 101 may transmit the first text to the server 108 andtransmit information for identifying the character being currently usedto the server 108. The first text and the information for identifyingthe character being used may be transmitted simultaneously orseparately. In operation 2403, the external electronic device 102 maytransmit information about a second graphic object corresponding to asecond user and a second text to the server 108. The external electronicdevice 102 may also transmit the second text to the server 108 andtransmit information for identifying the character being used by thesecond user of the external electronic device 102 to the server 108.

In operation 2405, the server 108 may analyze the first user's emotionalstate based on the first text and the second user's emotional statebased on the second text. In operation 2407, the server 108 may identifythe first sub object based on the first user's emotional state andinformation about the first graphic object, e.g., the characteridentification information. Further, the server 108 may identify theidentified second sub object based on the second user's emotional stateand information about the second graphic object, e.g., characteridentification information. According to an embodiment, even uponidentifying the first sub object, the server 108 may additionally usethe second user's emotional state, and even upon identifying the secondsub object, the server 108 may additionally use the first user'semotional state. The server 108 may identify information associated withmerging the first sub object and the second sub object. For example,upon merging, the server 108 may identify, for example, and withoutlimitation, at least one of the size, position, property, or the like,of each of the first sub object and the second sub object. In operation2409, the server 108 may transmit the merging-associated information tothe electronic device 101. In operation 2411, the server 108 maytransmit the merging-associated information to the external electronicdevice 102. In operation 2413, the electronic device 101 may provide agraphic object which results from merging the first sub object and thesecond sub object. For example, the electronic device 101 may merge anddisplay the first sub object and the second sub object by configuringpre-stored sub objects based on the merging-associated information. Evenwhere no resource for the second sub object is stored, the electronicdevice 101 may send a request for the resource to the server 108,receive the resource, and display a merged graphic object. In operation2415, the external electronic device 102 may provide a graphic objectwhich results from merging the first sub object and the second subobject.

For example, as shown in FIG. 25, the electronic device 101 may displaya first text 2510 and a first graphic object 2511 corresponding thereto.The first graphic object 2511 may correspond to the character being usedby the first user. The external electronic device 102 may transmit thesecond text 2520 to the server 108 and, corresponding thereto, theserver 108 may transmit merged information of the first sub object 2521and the second sub object 2522 or the merged graphic object to theelectronic device 101 and the external electronic device 102.Accordingly, the electronic device 101 may display a merged graphicobject of the second text 2520, the first sub object 2521, and thesecond sub object 2522. For example, the first sub object 2521 may beone selected to correspond to the first user's emotional state whilecorresponding to the character being used by the first user. The secondsub object 2522 may be one selected to correspond to the second user'semotional state while corresponding to the character being used by thesecond user. The electronic device 101 may obtain a third text 2530 andtransmit the third text 2530 to the server 108. In this case, unless thecharacter is changed, the electronic device 101 may only transmit thethird text 2530 to the server 108. The server 108 may analyze the thirdtext 2530 and identify the first user's emotional state. The server 108may identify a third sub object 2532 and a fourth sub object 2531corresponding to the first user's emotional state and the sub object'semotional state and transmit information about the merging of both thesub objects to the electronic device 101. The electronic device 101 maymerge and display the third sub object 2532 and the fourth sub object2531 based on the merging-related information received from the server108. According to an embodiment, the server 108 may be configured totransmit directly the merged graphic object, rather than themerging-related information, to the terminal device (e.g., theelectronic device 101). In this case, the electronic device 101 mayimmediately display the graphic object created by merging the subobjects received from the server 108.

FIG. 26 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment of the disclosure. Theembodiment shown in FIG. 26 is described in greater detail withreference to FIG. 27. FIG. 27 is a diagram illustrating an examplescreen displayed on an electronic device according to an embodiment.

In operation 2601, the electronic device 101 may receive a selection ofa first text and a first graphic object. In operation 2603, theelectronic device 101 may transmit the first text and the first graphicobject to the server 108 that may then be delivered to the externalelectronic device 102. The electronic device 101 may display thetransmitted text 2710 and the first graphic object 2711 on aconversation window. In operation 2605, the external electronic device102 may transmit a second text and a second graphic object to theelectronic device 101. The electronic device 101 may display thereceived second text 2720 and the second graphic object 2721. Inoperation 2607, the electronic device 101 may identify the first user'semotional state based on the first text 2710 and the second user'semotional state based on the second text 2720. In operation 2609, theelectronic device 101 may provide at least one recommendation graphicobject associated with a combination of a first sub object correspondingto the first user's emotional state and the first graphic object and asecond sub object corresponding to the second user's emotional state andthe second graphic object. For example, as shown in FIG. 27, even beforeany input is entered to the text window, the electronic device 101 maydisplay at least one recommendation graphic object 2723 and 2724 byvarious combinations of the first sub object corresponding to the firstuser's emotional state and corresponding to the first graphic object2711 and the second sub object corresponding to the second user'semotional state and corresponding to the character of the second graphicobject 2721. In operation 2611, the electronic device 101 may transmit agraphic object selected from among the recommendation graphic objects.Or, the electronic device 101 may receive an additional text andtransmit the selected graphic object along with the text.

FIG. 28 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment of the disclosure. Theembodiment related to FIG. 28 is described in greater detail withreference to FIG. 29. FIG. 29 is a diagram illustrating an examplescreen displayed on an electronic device and an example captured imageaccording to an embodiment.

According to an embodiment, in operation 2801, the electronic device 101may receive a selection of a first text and a first graphic object. Inoperation 2803, the electronic device 101 may transmit the first textand the first graphic object. In operation 2805, the electronic device101 may receive a second text and a second graphic object from theexternal electronic device 102. The electronic device 101 may displaythe received second text 2910 and second graphic object 2911 as shownin, e.g., FIG. 29. In operation 2807, the electronic device 101 mayobtain an image 2940 captured of the first user through, e.g., a camera.In operation 2809, the electronic device 101 may identify the firstuser's emotional state based on the first text and the image analysisresult and identify the second user's emotional state based on thesecond text. In operation 2811, the electronic device 101 may provide atleast one recommendation graphic object associated with a combination ofa first sub object corresponding to the first user's emotional state andthe first graphic object and a second sub object corresponding to thesecond user's emotional state and the second graphic object. Forexample, as shown in FIG. 29, the electronic device 101 may display thesecond sub object 2951 corresponding to the second user's emotionalstate. Further, the electronic device 101 may also display, as arecommendation graphic object, the first sub object 2952 correspondingto the first user's emotional state identified based on the imageanalysis result. In operation 2813, the electronic device 101 maytransmit a graphic object selected from among the recommendation graphicobjects.

FIG. 30 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment of the disclosure.

In operation 3001, the electronic device 101 may receive a selection ofa first text and a first graphic object. In operation 3003, theelectronic device 101 may transmit the first text and the first graphicobject. In operation 3005, the electronic device 101 may receive asecond text and a second graphic object. In operation 3007, theelectronic device 101 may identify the first user's emotional statebased on the first text and the second user's emotional state based onthe second text. In operation 3009, the electronic device 101 mayidentify the type of the first graphic object and the type of the secondgraphic object. The electronic device 101 may identify identificationinformation about the character used by the first user andidentification information about the character used by the second user.In operation 3011, the electronic device 101 may identify at least onefirst candidate sub object corresponding to the first user's emotionalstate among a plurality of first sub objects included in the type of thefirst graphic object and at least one second candidate sub objectcorresponding to the second user's emotional state among a plurality ofsecond sub objects included in the type of the second graphic object. Inother words, the electronic device 101 may identify a plurality ofcandidate sub objects set per character. In operation 3013, theelectronic device 101 may provide at least one recommendation graphicobject based on a combination of the first candidate sub objects and thesecond candidate sub objects. The electronic device 101 may transmit oneselected among the recommendation graphic objects to the externalelectronic device 102.

FIG. 31A is a flowchart illustrating an example method of operating anelectronic device according to an embodiment. FIG. 31B is a flowchartillustrating an example method of operating an electronic deviceaccording to an embodiment. The embodiment of FIGS. 31A and 31B isdescribed in greater detail with reference to FIGS. 32A, 32B and 32C.FIG. 32A is a diagram illustrating an example screen displayed on anelectronic device according to an embodiment. FIG. 32B is a diagramillustrating an example screen displayed on an electronic deviceaccording to an embodiment. FIG. 32C is a diagram illustrating anexample screen displayed on an electronic device according to anembodiment.

According to an embodiment, in operation 3111, a first electronic device1101 may receive a selection of a first text and a first graphic object.In operation 3113, the first electronic device 1101 may transmit thefirst text and the first graphic object to the server 1104. The firstelectronic device 1101 is presumed to have attended a group conversationsession of three or more users. In other words, the first electronicdevice 1101, a second electronic device 1102, and a third electronicdevice 1103 may attend the group conversation session.

In operation 3115, the server 1104 may identify information about thefirst text and the first graphic object. In operation 3117, the server1104 may transmit the information about the first text and the firstgraphic object to the electronic devices 1101, 1102 and 1103 attendingthe group conversation session. The first electronic device 1101, thesecond electronic device 1102, and the third electronic device 1103 mayprovide the first text and the first graphic object in operations 3119,3121, and 3123. In operation 3125, the second electronic device 1102 mayreceive a selection of a second text and a second graphic object. Inoperation 3127, the second electronic device 1102 may transmit thesecond text and the second graphic object to the server 1104. Inoperation 3129, the server 1104 may analyze the first user's emotionalstate based on the first text and the second user's emotional statebased on the second text. In operation 3131, the server 1104 mayidentify the first sub graphic object corresponding to the first userbased on the first user's emotional state and the second user'semotional state and identify the second sub graphic object correspondingto the second user based on the first user's emotional state and thesecond user's emotional state. In operation 3133, the server 1104 maytransmit information associated with the first sub graphic object andthe second sub graphic object to the electronic devices 1101, 1102, 1103attending the group conversation session. In operations 3135, 3137, and3139, the first electronic device 1101, the second electronic device1102, and the third electronic device 1103 may provide a synthesizedgraphic object based on the information associated with the first subgraphic object and the second sub graphic object. For example, as shownin FIG. 32A, the third electronic device 1103 may display the first text3202 and the first graphic object 3201 obtained from the firstelectronic device 1101. Further, the server 1004 may provide the secondtext 3205 obtained from the second electronic device 1102 and a graphicobject composed of the sub object 3204 corresponding to the characteridentified from the second electronic device 1102 and the sub object3203 corresponding to the character identified from the first electronicdevice 1101 to the third electronic device 1103, and the thirdelectronic device 1103 may display the graphic object including theplurality of sub objects 3203 and 3204 and the second text 3205. The subobject 3204 may be determined by the server 1104 based on the emotionalstate of the user of the second electronic device 1102. The server 1004may provide the third text 3209 obtained from a fourth electronic device(not shown) and a graphic object comprising the sub object 3208corresponding to the character identified from the fourth electronicdevice (not shown), the sub object 3206 corresponding to the characteridentified from the first electronic device 1101, and the sub object3207 corresponding to the character identified from the secondelectronic device 1102 to the third electronic device 1103, and thethird electronic device 1103 may display the graphic object constitutedof the plurality of sub objects 3206, 3207, and 3208 and the third text3209. Even where no graphic object is designated from the secondelectronic device 1102 and only the text 3205 is entered to the secondelectronic device 1102, the server 1104 may identify the sub object 3204based on the emotional state of the second user of the second electronicdevice 1102 and the character corresponding to the second electronicdevice 1102 and provide the same to the second electronic device 1102.As shown in FIG. 32A, the server 1104 may vary the position of the subobject 3207 in the overall graphic object to add the sub object 3208.The third electronic device 1103 may receive the text 3211 and transmitthe text 3211 to the server 1104. The third electronic device 1103 maydisplay the text 3211 and the designated character 3210. Or, as shown inFIG. 32B, the second electronic device 1102 may display objects 3221 and3222 to select whether to add a second graphic object in associationwith the graphic object 3201 while displaying the first text 3202 andthe graphic object 3201 identified from the first electronic device1101. Where an object 3221 to indicate “add” is designated, the secondelectronic device 1102 may transfer the designation of add object to theserver 1104 and, corresponding thereto, the server 1104 may add a subobject based on the user's emotional state identified from the text fromthe second electronic device 1102 and transfer the added sub object tothe electronic devices in the session. In this case, there may beadditionally provided a user interface for adding the type or positionof the sub object to be added, and the second electronic device 1102 maytransfer information obtained through the user interface to the server1104. The server 1104 may determine at least one of the type or positionof the sub object based on the received information. Where an object3222 to indicate “end”, the server 1104 may only transfer the text fromthe second electronic device 1102 to the electronic devices in thesession, corresponding to the designation. According to an embodiment,where all the participants in the session are identified to have addedthe sub object, the server 1104 may automatically terminate the subobject add. Or, upon identifying that the number of sub objects added isa threshold or more, the server 1104 may be configured to stop adding asub object. As shown in FIG. 32C, the third electronic device 1103 mayreceive a terminate add command. Where the third electronic device 1103transfers the terminate add command to the server 1104, the server 1104may display together a graphic object reflecting a sub object 3274 toindicate “complete” to the prior-added sub objects 3271, 3272, and 3273,the text 3276 entered from the third electronic device 1103, and a photo3275 of the user of the third electronic device 1103.

FIG. 33 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment of the disclosure. Theembodiment related to FIG. 33 is described in greater detail withreference to FIG. 34. FIG. 34 is a diagram illustrating an examplescreen displayed on an electronic device according to an embodiment.

In operation 3301, the electronic device 101 may transmit a first textto the server 108. In operation 3303, the external electronic device 102may transmit a second text to the server 108. In operation 3305, theserver 108 may analyze the first user's emotional state based on thefirst text and the second user's emotional state based on the secondtext. In operation 3307, the server 108 may identify informationassociated with a merging of a first sub object identified based on thefirst user's emotional state and the first user's property informationand a second sub object identified based on the second user's emotionalstate and the second user's property information. As set forth above,the user's property information may be information associated with theidentified user, such as, for example, and without limitation, theuser's age, gender, personality, or the like. The server 108 mayidentify the sub object based on the user's property along with theuser's emotional state. Or, the server 108 may identify the user'semotional state based on a result of text analysis and the user'sproperty and may identify the sub object based on the user's emotionalstate. In operation 3309, the server 108 may transmit the mergingassociated information associated with the sub objects to the electronicdevice 101. In operation 3311, the server 104 may transmit the mergingassociated information associated with the sub objects to the externalelectronic device 102. In operation 3313, the electronic device 101 mayprovide a GUI which is a merged one of the first sub object and thesecond sub object based on the received merging-associated information.In operation 3315, the external electronic device 102 may provide a GUIwhich is a merged one of the first sub object and the second sub objectbased on the received merging-associated information. For example, asshown in FIG. 34, the electronic device 101 may transmit the first text3402 to the server 108 and, as a result of analysis of the first text3402, the server 108 may identify the type of the first user's emotionas “sad” and transmit the same to the electronic device 101. Theelectronic device 101 may display the graphic object 3401 based on theidentified type of emotion. The server 108 may receive the second text3405 from the second user's electronic device, identify the type of theuser's emotion corresponding thereto as “encouragement” and, basedthereupon, transmit a graphic object including the sub object 3403corresponding to the first user and the sub object 3404 corresponding tothe second user to the second user's electronic device. Where the seconduser's electronic device issues a command to transmit the graphicobject, the electronic device 101 may receive the graphic object and thesecond text 3405. The electronic device 101 may display the sub objects3404 and 3403 of the graphic object and the second text 3405. The thirduser's electronic device may enter the third text 3408. The server 108may receive the third text 3408 and identify the third user's type ofemotion as “encouragement”. Additionally, the server 108 may identifythe gender which is the first user's property and the gender which isthe second user's property and identify the sub object corresponding tothe first user as the sub object 3406 corresponding to the type ofemotion, “embarrassed”. The server 108 may identify the sub object 3407corresponding to the third user's type of emotion, “encouragement”, andtransmit the merging-related information to the third user's electronicdevice. Upon receiving the graphic object and the third text from thethird user's electronic device, the electronic device 101 may displaythe sub objects 3406 and 3407 and the third text 3408.

FIG. 35 is a flowchart illustrating an example method of operating anelectronic device according to an embodiment. The embodiment of FIG. 35is described in greater detail in connection with FIG. 36. FIG. 36 is adiagram illustrating an example screen displayed on an electronic deviceaccording to an embodiment.

According to an embodiment, in operation 3501, the external electronicdevice 102 may create the second user's video and a first graphic objectvideo corresponding to the second user's video. For example, theexternal electronic device 102 may capture the user's face, identifyfeature points of the user's face, and create the first graphic objectvideo copying the user's facial expressions based on the positions ofand changes in the feature points. In operation 3503, the externalelectronic device 102 may transmit the first graphic object video to theelectronic device 101. The electronic device 101 may play the firstgraphic object video in operation 3505. For example, as shown in FIG.36, the electronic device 101 may play the received first graphic videoobject 3611. Further, the electronic device 101 may also display thetext 3612 received from the external electronic device 102. In operation3507, the electronic device 101 may create the first user's video and asecond graphic object video corresponding to the first user's videofile. For example, the electronic device 101 may create the first user'svideo file by capturing the user using the camera and create a secondgraphic video object copying the expressions. In operation 3509, theelectronic device 101 may create and transmit a synthesized video of thefirst graphic object video and the second graphic object video. Forexample, the electronic device 101 may temporally synchronize the firstgraphic object video and the second graphic object video and arrangethem spatially adjacent each other, thereby creating the synthesizedvideo. In operation 3511, the external electronic device 102 may playthe synthesized video. The electronic device 101 may play thesynthesized video including the plurality of sub videos 3621 and 3622.The electronic device 101 may transmit a text received and display thereceived text 3623. The electronic device 101 may display a text 3624additionally received.

FIG. 37A is a diagram illustrating example screens displayed on anelectronic device according to an embodiment. FIG. 37B is a diagramillustrating example screens displayed on an electronic device accordingto an embodiment. Upon receiving a text 3703, the electronic device 101may display a graphic object 3701 corresponding thereto. Upon receivinga text 3713 as partially modified, the electronic device 101 may displaya graphic object 3711 resulting from modifying at least some of theresources. As shown in FIG. 37B, the electronic device 101 may display agraphic object 3723 corresponding to a text 3721. The electronic device101 may receive a text 3731 in a different position on the screen anddisplay a graphic object 3733 corresponding to the text 3731 in theposition.

FIG. 38 is a block diagram illustrating an example processor accordingto an embodiment.

Referring to FIG. 38, according to an embodiment, a processor (e.g.,including processing circuitry) 3800 may include a data learning unit(e.g., including processing circuitry and/or program elements) 3810 anda data recognizing unit (e.g., including processing circuitry and/orprogram elements) 3820. The processor 3800 may be included in the server108 or the electronic device 101.

The data learning unit 3810 may include various processing circuitryand/or program elements and learn references for determining at leastone of the type and degree of emotion or the type of actioncorresponding to an obtained text. The data learning unit 3810 may learnthe references for determining at least one of the type and degree ofemotion or the type of action corresponding to the text by obtaininglearning data to be used for learning and applying the obtained learningdata to a data recognition model to be described below.

For example, the data learning unit 3810 may use statistical data forinformation about at least one of the type and degree of emotion or thetype of action corresponding to the text recommended by the server 108or the electronic device 101 and the text as the learning data forlearning the references.

The data learning unit 3810 may train the data recognition model basedon the learning data through unsupervised learning.

The data learning unit 3810 may train or update the data recognitionmodel using the user's feedback as per the results of the recognition.Specifically, the data learning unit 3810 may train the data recognitionmodel such that among at least one of the type and degree of emotion orthe type of action corresponding to the text recommended by the server108 or the electronic device 101, the one selected by the user isassigned a larger weight.

The data recognizing unit 3820 may include various processing circuitryand/or program elements and determine at least one of the type anddegree of emotion or the type of action corresponding to the textentered from the user based on the data. For example, the datarecognizing unit 3820 may determine at least one of the type and degreeof emotion or the type of action corresponding to the text using theresults of learning. The data recognizing unit 3820 may obtainpredetermined data as per preset references by learning and use the datarecognition model using the obtained data as an input value. By usingthe same, the data recognizing unit 3820 may determine at least one ofthe type and degree of emotion or the type of action corresponding tothe text to be recommended based on the predetermined data. Theresultant value output by the data recognition model using the obtaineddata as an input value may be used to update the data recognition model.

For example, the data recognizing unit 3820 may obtain a text asrecognition data. The data recognizing unit 3820 may obtain at least oneof the type and degree of emotion or the type of action corresponding tothe text by applying the obtained data to the data recognition model.

At least part of the data learning unit 3810 and at least part of thedata recognizing unit 3820 may be produced in at least one hardware chipthat may then be equipped in the electronic device 101 or the server108. For example, at least one of the data learning unit 3810 and thedata recognizing unit 3820 may be formed in a dedicated hardware chipfor artificial intelligence (AI) or in part of an existinggeneral-purpose processor (e.g., a central processing unit (CPU) orapplication processor) or a graphic dedicated processor (e.g., graphicsprocessing unit (GPU)) and be equipped in various electronic devices. Inthis case, the dedicated hardware chip for AI may be a dedicatedprocessor specified for probability computation, which may quicklyprocess AI computation tasks, e.g., machine learning, with a betterparallel processing performance than existing general-purposeprocessors.

The data learning unit 3810 and the data recognizing unit 3820 may beequipped in one electronic device 101 or the server 108 or in separateelectronic devices. For example, one of the data learning unit 3810 andthe data recognizing unit 3820 may be included in the electronic device101, and the other in the server 108. The data learning unit 3810 andthe data recognizing unit 3820 may wiredly or wirelessly provide modelinformation established by the data learning unit 3810 to the datarecognizing unit 3820, and data entered to the data recognizing unit3820, as additional learning data, may be provided to the data learningunit 3810.

At least part of the data learning unit 3810 and at least part of thedata recognizing unit 3820 may be implemented in at least one softwaremodule. Where at least one of the data learning unit 3810 and the datarecognizing unit 3820 is implemented in a software module (or a programmodule including instructions), the software module may be stored in anon-transitory computer-readable recording medium. At least one softwaremodule may be provided by an operating system (OS) or by a predeterminedapplication. Some of the at least one software module may be provided bythe OS, and others may be provided by the predetermined application.

FIG. 39 is a block diagram illustrating an example data learning unit3810 according to an embodiment.

Referring to FIG. 39, according to an embodiment, the data learning unit3810 may include a data obtaining unit (e.g., including processingcircuitry and/or program elements) 3810-1, a pre-processor (e.g.,including processing circuitry and/or program elements) 3810-2, alearning data selecting unit (e.g., including processing circuitryand/or program elements) 3810-3, a model learning unit (e.g., includingprocessing circuitry and/or program elements) 3810-4, and a modelevaluating unit (e.g., including processing circuitry and/or programelements) 3810-5. According to an embodiment, the data learning unit3810 may include the data obtaining unit 3810-1 and the model learningunit 3810-4 and may optionally include at least one of the pre-processor3810-2, the learning data selecting unit 3810-3, and the modelevaluating unit 3810-5.

The data obtaining unit 3810-1 may include various processing circuitryand/or program elements and obtain learning data necessary to determineat least one of the type and degree of emotion or the type of actioncorresponding to a text.

The data obtaining unit 3810-1 may obtain, e.g., a text. Further, thedata obtaining unit 3810-1 may obtain data selected by the user among atleast one of the type and degree of emotion or the type of actioncorresponding to a text recommended by the server 108 or the electronicdevice 101.

The pre-processor 3810-2 may include various processing circuitry and/orprogram elements and may pre-process the obtained text. Thepre-processor 3810-2 may process the obtained data into a preset formatto allow the model learning unit 3810-4 to use the obtained data forlearning at least one of the type and degree of emotion or the type ofaction corresponding to the text.

The learning data selecting unit 3810-3 may include various processingcircuitry and/or program elements and may select learning data necessaryfor learning among the pre-processed data. The selected learning datamay be provided to the model learning unit 3810-4. The learning dataselecting unit 3810-3 may determine at least one of the type and degreeof emotion or the type of action corresponding to a text to berecommended. The learning data selecting unit 3810-3 may also selectlearning data as per references preset by learning by the model learningunit 3810-4 described below.

The model learning unit 3810-4 may include various processing circuitryand/or program elements and may determine at least one of the type anddegree of emotion or the type of action corresponding to the text basedon the learning data selected by the learning data selecting unit 3810-3or obtained through the data obtaining unit 3810-1.

The model learning unit 3810-4 may train, with learning data, the datarecognition model used to determine at least one of the type and degreeof emotion or the type of action corresponding to the text. In thiscase, the data recognition model may be a model previously built up. Forexample, the data recognition model may be a model previously built upby receiving basic learning data (e.g., sample data).

The data recognition model may be established given, e.g., the field ofapplication, the purpose of learning, or the computation performance ofthe device. The data recognition model may be designed to mimic thehuman brain on the computer. The data recognition model may include aplurality of weight-assigned network nodes mimicking the neurons of thehuman neural network. The plurality of network nodes may formconnections to mimic the synaptic activity of neurons to exchangesignals through the synapses. The data recognition model may include,e.g., a neural network model or a deep learning model as an advancedversion of the neural network model. In the deep learning model, aplurality of network nodes may be positioned at different depths (orlayers) to exchange data as per convolution connections. For example,the deep neural network (DNN), recurrent neural network (RNN),bidirectional recurrent deep neural network (BRDNN) or such types ofmodels may be used as the data recognition model, but not limitedthereto.

According to an embodiment, where there are previously built up aplurality of data recognition models, the model learning unit 3810-4 maydetermine the data recognition model with the larger relevance betweeninput learning data and basic learning data as the data recognitionmodel for training. In this case, the basic learning data may bepreviously classified per data type, and the data recognition model maypreviously be established per data type. For example, the basic learningdata may previously be classified by various references, such as theregion and time that the learning data was created, the size, genre, andcreator of the learning data, and the type of objects in the learningdata.

The model learning unit 3810-4 may train the data recognition modelusing learning algorithms including, e.g., error back-propagation orgradient descent.

The model learning unit 3810-4 may train the data recognition modelthrough, e.g., supervised learning using at least part of the learningdata as references for determination.

The model learning unit 3810-4 may train the data recognition modelthrough unsupervised learning that discovers references for determiningat least one of the type and degree of emotion or the type of actioncorresponding to the text by self-learning at least one of the type anddegree of emotion or the type of action corresponding to the text usinglearning data without supervision. Or, the model learning unit 3810-4may train the data recognition model using feedback information that isat least one of the type and degree of emotion or the type of actioncorresponding to the text selected by the user.

The model learning unit 3810-4 may also train the data recognition modelthrough reinforcement learning that uses, e.g., a determination of theuser's intention as per learning, the provision of associatedinformation, and feedback as to whether the result of recommending areplacement action is correct.

Where the data recognition model is trained, the model learning unit3810-4 may store the trained data recognition model. In this case, themodel learning unit 3810-4 may store the trained data recognition modelin the memory of the electronic device (e.g., the server 108 or theelectronic device 101 ) including the data recognizing unit 3820. Or,the model learning unit 3810-4 may store the trained data recognitionmodel in the memory (e.g., the server 108 or the electronic device 101)of the electronic device including the data recognizing unit 3820 asdescribed below. Or, the model learning unit 3810-4 may store thetrained data recognition model in a memory of a server connected withthe electronic device (e.g., the server 108 or the electronic device101) via a wired or wireless network.

In this case, the memory storing the trained data recognition model mayalso store, e.g., commands or data related to at least one othercomponent of the electronic device. The memory may store software and/ora program. The program may include, e.g., a kernel, middleware, anapplication programming interface (API), and/or an application program(or an application).

The model evaluating unit 3810-5 may include various processingcircuitry and/or program elements and may enter evaluation data to thedata recognition model and, where a result of recognition output fromthe evaluation data fails to meet a predetermined reference, enable themodel learning unit 3810-4 to retrain the data recognition model. Inthis case, the evaluation data may be preset data to evaluate the datarecognition model.

For example, where the trained data recognition model shows incorrectresults of recognition that exceed a preset threshold number orproportion in relation to the evaluation data, the model evaluating unit3810-5 may evaluate that the data recognition model fails to meet apredetermined reference. For example, where the predetermined referenceis defined as a 2% proportion, and the trained data recognition modeloutputs wrong results of recognition for more than 20 pieces ofevaluation data among a total of 1,000 pieces of evaluation data, themodel evaluating unit 3810-5 may evaluate that the trained datarecognition model is inappropriate.

Meanwhile, where there are a plurality of trained data recognitionmodels, the model evaluating unit 3810-5 may evaluate whether eachtrained data recognition model meets a predetermined reference anddetermines the one meeting the predetermined reference as a final datarecognition model. In this case, where there are a plurality of modelsmeeting the predetermined reference, the model evaluating unit 3810-5may determine any one or a predetermined number of models as final datarecognition models in order of higher evaluation grade.

Meanwhile, at least one of the data obtaining unit 3810-1, thepre-processor 3810-2, the learning data selecting unit 3810-3, the modellearning unit 3810-4, and the model evaluating unit 3810-5 in the datalearning unit 3810 may be produced in at least one hardware chip thatmay then be equipped in the electronic device. For example, at least oneof the data obtaining unit 3810-1, the pre-processor 3810-2, thelearning data selecting unit 3810-3, the model learning unit 3810-4, andthe model evaluating unit 3810-5 may be formed in a dedicated hardwarechip for artificial intelligence (AI) or in part of an existinggeneral-purpose processor (e.g., a central processing unit (CPU) orapplication processor) or a graphic dedicated processor (e.g., graphicsprocessing unit (GPU)) and be equipped in various electronic devices.

Further, the data obtaining unit 3810-1, the pre-processor 3810-2, thelearning data selecting unit 3810-3, the model learning unit 3810-4, andthe model evaluating unit 3810-5 may be equipped in one electronicdevice or each may be equipped in a respective one of separateelectronic devices. For example, some of the data obtaining unit 3810-1,the pre-processor 3810-2, the learning data selecting unit 3810-3, themodel learning unit 3810-4, and the model evaluating unit 3810-5 may beincluded in the electronic device 101, and the rest in the server 108.

Further, at least one of the data obtaining unit 3810-1, thepre-processor 3810-2, the learning data selecting unit 3810-3, the modellearning unit 3810-4, and the model evaluating unit 3810-5 may beimplemented in a software module. Where at least one of the dataobtaining unit 3810-1, the pre-processor 3810-2, the learning dataselecting unit 3810-3, the model learning unit 3810-4, and the modelevaluating unit 3810-5 is implemented in a software module (or a programmodule including instructions), the software module may be stored in anon-transitory readable recording medium. In this case, at least onesoftware module may be provided by an operating system (OS) or by apredetermined application. Some of the at least one software module maybe provided by the OS, and others may be provided by the predeterminedapplication.

The processor 3800 (or a controller) may use various data recognitionmodels and determine the user's intention, provide associatedinformation, and efficiently learn references for recommendingreplacement actions in various manners through the data recognitionmodels.

FIG. 40 is a block diagram illustrating an example data recognizing unit3820 according to an embodiment.

Referring to FIG. 40, according to an embodiment, the data recognizingunit (e.g., including processing circuitry and/or program elements) 3820may include a data obtaining unit data obtaining unit (e.g., includingprocessing circuitry and/or program elements) 3820-1, a pre-processor(e.g., including processing circuitry and/or program elements) 3820-2, arecognition data selecting unit (e.g., including processing circuitryand/or program elements) 3820-3, a recognition result providing unit(e.g., including processing circuitry and/or program elements) 3820-4,and a model updating unit (e.g., including processing circuitry and/orprogram elements) 3820-5. According to an embodiment, the datarecognizing unit 3820 may necessarily include the data obtaining unit3820-1 and the recognition result providing unit 3820-4 and mayoptionally include at least one of the pre-processor 3820-2, therecognition data selecting unit 3820-3, and the model updating unit3820-5.

The data obtaining unit 3820-1 may include various processing circuitryand/or program elements and obtain data necessary to determine at leastone of the type and degree of emotion or the type of actioncorresponding to a text.

The pre-processor 3820-2 may include various processin circuitry and/orprogram elements and may determine and pre-process at least one of thetype and degree of emotion or the type of action corresponding to atext. The pre-processor 3820-2 may determine at least one of the typeand degree of emotion or the type of action corresponding to the text tobe recommended by the recognition result providing unit 3820-4 asdescribed below and process the obtained data in a preset format. Forexample, the data obtaining unit 3820-1 may obtain a text enteredthrough the electronic device 101. The data obtaining unit 3820-1 mayobtain context information related to the user or the electronic device101. For example, the context information may be information created bythe electronic device 101 or information received from an externaldevice.

The recognition data selecting unit 3820-3 may include variousprocessing circuitry and/or program elements and may determine at leastone of the type and degree of emotion or the type of actioncorresponding to the text among the pre-processed recognition data andselect recognition data. The selected recognition data may be providedto the recognition result providing unit 3820-4. The recognition dataselecting unit 3820-3 may select the whole or part of the pre-processedrecognition data as per at least one of the type and degree of emotionor the type of action corresponding to the text. The recognition dataselecting unit 3820-3 may also select recognition data as per referencespreset by learning by the model learning unit 3810-4 described below.

The recognition result providing unit 3820-4 may include variousprocessing circuitry and/or program elements and may determine at leastone of the type and degree of emotion or the type of actioncorresponding to the text to be recommended by applying the selectedrecognition data to the data recognition model. The recognition resultproviding unit 3820-4 may provide the results of recognition as per thepurposes of recognition of data. The recognition result providing unit3820-4 may apply the recognition data obtained through the dataobtaining unit 3820-1 or the recognition data selected by therecognition data selecting unit 3820-3, as input values, to the datarecognition model. The recognition result providing unit 3820-4 mayobtain at least one of the type and degree of emotion or the type ofaction corresponding to the text as a result of applying to the datarecognition model.

The model updating unit 3820-5 may include various processing circuitryand/or program elements and may allow the data recognition model to beupdated based on evaluations on the results of recognition provided bythe recognition result providing unit 3820-4. For example, the modelupdating unit 3820-5 may provide the results of recognition providedfrom the recognition result providing unit 3820-4 to the model learningunit 3810-4, allowing the model learning unit 3810-4 to update the datarecognition model.

For example, the evaluations on the results of recognition may be theuser's feedback information as per at least one of the type and degreeof emotion or the type of action corresponding to the text recommendedfor the user by the server 108 or the electronic device 101. As anexample, among at least one of the type and degree of emotion or thetype of action corresponding to the text recommended by the server 108or the electronic device 101, at least one of the type and degree ofemotion or the type of action corresponding to the text selected by theuser, as user information, may be used as learning data to update thedata recognition model.

At least one of the data obtaining unit 3820-1, the pre-processor3820-2, the recognition data selecting unit 3820-3, the recognitionresult providing unit 3820-4, and the model updating unit 3820-5 in thedata recognizing unit 3820 may be produced in at least one hardware chipthat may then be equipped in the electronic device. For example, atleast one of the data obtaining unit 3820-1, the pre-processor 3820-2,the recognition data selecting unit 3820-3, the recognition resultproviding unit 3820-4, and the model updating unit 3820-5 may be formedin a dedicated hardware chip for artificial intelligence (AI) or in partof an existing general-purpose processor (e.g., a central processingunit (CPU) or application processor) or a graphic dedicated processor(e.g., graphics processing unit (GPU)) and be equipped in variouselectronic devices.

Further, the data obtaining unit 3820-1, the pre-processor 3820-2, therecognition data selecting unit 3820-3, the recognition result providingunit 3820-4, and the model updating unit 3820-5 may be equipped in oneelectronic device or each may be equipped in a respective one ofseparate electronic devices. For example, some of the data obtainingunit 3820-1, the pre-processor 3820-2, the recognition data selectingunit 3820-3, the recognition result providing unit 3820-4, and the modelupdating unit 3820-5 may be included in the electronic device 101, andthe rest in the server.

Further, at least one of the data obtaining unit 3820-1, thepre-processor 3820-2, the recognition data selecting unit 3820-3, therecognition result providing unit 3820-4, and the model updating unit3820-5 may be implemented in a software module. Where at least one ofthe data obtaining unit 3820-1, the pre-processor 3820-2, therecognition data selecting unit 3820-3, the recognition result providingunit 3820-4, and the model updating unit 3820-5 is implemented in asoftware module (or a program module including instructions), thesoftware module may be stored in a non-transitory readable recordingmedium. In this case, at least one software module may be provided by anoperating system (OS) or by a predetermined application. Some of the atleast one software module may be provided by the OS, and others may beprovided by the predetermined application.

The server 108 or the electronic device 101 may provide the user with aservice suited for the user's intention using the data recognition modelto which the learned results have been applied.

The electronic device according to various embodiments may be one ofvarious types of electronic devices. The electronic devices may includeat least one of, e.g., a portable communication device (e.g., asmartphone), a computer device, a portable multimedia device, a portablemedical device, a camera, a wearable device, or a home appliance.According to an embodiment of the disclosure, the electronic device isnot limited to the above-listed embodiments.

It should be appreciated that various example embodiments of thedisclosure and the terms used therein are not intended to limit thetechniques set forth herein to particular embodiments and that variouschanges, equivalents, and/or replacements therefor also fall within thescope of the disclosure. The same or similar reference denotations maybe used to refer to the same or similar elements throughout thespecification and the drawings. It is to be understood that the singularforms “a”, “an”, and “the” include plural references unless the contextclearly dictates otherwise. As used herein, the term “A or B”, “at leastone of A and/or B”, “A, B, or C”, or “at least one of A, B, and/or C”may include all possible combinations of the enumerated items. As usedherein, the terms “1st” or “first” and “2nd” or “second” may be used toidentify corresponding components regardless of importance and/or orderand are used to distinguish a component from another without limitingthe components. It will be understood that when an element (e.g., afirst element) is referred to as being (operatively or communicatively)“coupled with/to”, or “connected with/to” another element (e.g., asecond element), it can be coupled or connected with/to the otherelement directly or via a third element.

As used herein, the term “module” includes a unit configured inhardware, software, or firmware or any combination thereof, and mayinterchangeably be used with other terms, e.g., “logic”, “logic block”,“part”, or “circuit”. A module may be a single integral part or aminimum unit or part for performing one or more functions. For example,the module may be configured in an application-specific integratedcircuit (ASIC).

Various embodiments as set forth herein may be implemented as software(e.g., the program) containing commands that are stored in a machine(e.g., computer)-readable storage medium (e.g., an internal memory) oran external memory. The machine may be a device that may invoke acommand stored in the storage medium and may be operated as per theinvoked command. The machine may include an electronic device (e.g., theelectronic device 101) according to embodiments disclosed herein. Whenthe command is executed by a processor (e.g., the processor 120), theprocessor may perform a function corresponding to the command on its ownor using other components under the control of the processor. Thecommand may contain a code that is generated or executed by a compileror an interpreter. The machine-readable storage medium may be providedin the form of a non-transitory storage medium. The term“non-transitory” refers to a storage medium that is tangible, but thisterm does not differentiate between where data is semi permanentlystored in the storage medium and where data is temporarily stored in thestorage medium.

According to an embodiment, a method according to various embodiments ofthe disclosure may be included and provided in a computer programproduct. The computer program products may be traded as commoditiesbetween sellers and buyers. The computer program product may bedistributed in the form of a machine-readable storage medium (e.g., acompact disc read only memory (CD-ROM)) or online through an applicationstore (e.g., Playstore'). When distributed online, at least part of thecomputer program product may be temporarily generated or at leasttemporarily stored in a storage medium, such as the manufacturer'sserver, a server of the application store, or a relay server.

A method according to various embodiments of the disclosure may beprovided in a computer program product.

Such computer program products may include software (S/W) programs orcomputer-readable storage media storing the S/W programs or productstraded between the seller and the buyer.

For example, the computer program products may include S/W program-typeproducts (e.g., downloadable applications (apps)) that areelectronically distributed through the electronic device 101 or themanufacturers of the electronic device or electronic markets (e.g.,Google Play Store or App Store). For electronic distribution, at leastpart of the S/W programs may be stored in storage media or temporarilygenerated. In this case, the storage media may be storage media of themanufacturer's or electronic market's server or the relay server.

According to various embodiments, each component (e.g., a module orprogram) may be configured of a single or multiple entities, and thevarious embodiments may exclude some of the above-described subcomponents or add other sub components. Some components (e.g., modulesor programs) may be integrated into a single entity that may thenperform the respective (pre-integration) functions of the components inthe same or similar manner. According to various embodiments, operationsperformed by modules, programs, or other components may be carried outsequentially, in parallel, repeatedly, or heuristically, or at leastsome operations may be executed in a different order or omitted, orother operations may be added.

As is apparent from the foregoing description, according to variousembodiments, there may be provided an electronic device and method foroperating the electronic device that may provide an image selected oredited depending on the type or degree of the user's emotion. Accordingto various embodiments, there may be provided an electronic device andmethod for operating the electronic device that may select or edit animage corresponding to the emotion among a plurality of users andprovide the selected or edited image. According to various embodiments,there may be provided an electronic device and method for operating theelectronic device that may merge images corresponding to a plurality ofusers into a single image and provided the merged image.

While the disclosure has been illustrated and described with referenceto certain various example embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade therein without departing from the spirit and scope of thedisclosure as defined, for example, by the appended claims and theirequivalents.

What is claimed is:
 1. An electronic device, comprising: an input devicecomprising input circuitry; a display device; a communication circuit;and at least one processor configured to control the electronic deviceto: receive a text through the input device, transmit first informationabout the text through the communication circuit to a server, receivesecond information associated with an image identified based on anemotional state of a first user, the emotional state of the first userbeing identified based on a result of analysis of the text and anemotional state of a second user conversing with the first user, anddisplay the image through the display device based on the secondinformation associated with the image.
 2. The electronic device of claim1, wherein the at least one processor is configured to control theelectronic device to: receive third information associated with a firstsub object corresponding to the emotional state of the first user andfourth information associated with a second sub object corresponding tothe emotional state of the second user, as at least part of receivingthe second information associated with the image, and display the imageincluding the first sub object and the second sub object, as at leastpart of displaying the image through the display device based on thesecond information associated with the image.
 3. The electronic deviceof claim 2, wherein the first sub object corresponds to a firstcharacter designated to be associated with the first user, and thesecond sub object corresponds to a second character designated to beassociated with the second user.
 4. The electronic device of claim 2,wherein the second information associated with the image includes firstidentification information identifying the first sub object and secondidentification information identifying the second sub object, andwherein the at least one processor is configured to control theelectronic device to: identify the first sub object corresponding to thefirst identification information among a plurality of stored images, andidentify the second sub object corresponding to the secondidentification information from among the plurality of images, as atleast part displaying the image through the display device based on thesecond information associated with the image.
 5. The electronic deviceof claim 2, wherein the second information associated with the imageincludes the emotional state of the first user and the emotional stateof the second user, and wherein the at least one processor is configuredto control the electronic device to: identify the first sub objectcorresponding to the emotional state of the first user from among aplurality of stored images, and identify the second sub objectcorresponding to the emotional state of the second user from among theplurality of images, as at least part of displaying the image throughthe display device based on the second information associated with theimage.
 6. The electronic device of claim 5, wherein the emotional stateof the first user includes a type of emotion of the first user and adegree of emotion of the first user, and the emotional state of thesecond user includes a type of emotion of the second user and a degreeof emotion of the second user, and wherein the at least one processor isconfigured to control the electronic device to: identify the first subobject corresponding to the type of emotion of the first user and thedegree of emotion of the first user from among a plurality of imagescorresponding to a first character designated to be associated with thefirst user. and identify the second sub object corresponding to the typeof emotion of the second user and the degree of emotion of the seconduser from among a plurality of images corresponding to a secondcharacter designated to be associated with the second user, as at leastpart of displaying the image through the display device based on thesecond information associated with the image.
 7. The electronic deviceof claim 2, wherein the second information associated with the imageincludes fifth information about a position of the first sub object andsixth information about a position of the second sub object, and whereinthe at least one processor is configured to control the electronicdevice to: place the first sub object in a first position in the imagebased on the fifth information, and place the second sub object in asecond position in the image based on the sixth information, as at leastpart of displaying the image based on the second information associatedwith the image.
 8. The electronic device of claim 1, wherein the atleast one processor is configured to control the electronic device to:receive the second information about the image including thirdinformation about the emotional state of the first user, the emotionalstate of the second user, and an action corresponding to the text, as atleast part of receiving the second information associated with theimage, and display the image corresponding to the second informationassociated with the image including the third information about theemotional state of the first user, the emotional state of the seconduser, and the action corresponding to the text, as at least part ofdisplaying the image through the display device based on the secondinformation associated with the image.
 9. The electronic device of claim1, wherein the second information associated with the image includesedit information about raw information, and wherein the at least oneprocessor is configured to control the electronic device to: edit afirst raw image designated to be associated with the first user usingthe edit information, and display the edited first raw image, as atleast part of displaying the image through the display device based onthe second information associated with the image.
 10. The electronicdevice of claim 9, wherein the at least one processor is configured tocontrol the electronic device to: select at least one of: a position, ashape, a color, and a changing speed of at least one object in theimage, or replace the at least one object in the image with anotherobject, as at least part of editing the first raw image using the editinformation.
 11. The electronic device of claim 1, further comprising acamera, wherein the at least one processor is configured to control theelectronic device to: transmit a first image obtained using the camerato the server, as at least part of transmitting the first informationabout the text to the server and, receive the second informationassociated with the image identified based on the result of analysis ofthe first image, the emotional state of the first user, and theemotional state of the second user, as at least part of receiving thesecond information associated with the image identified based on theemotional state of the first user and the emotional state of the seconduser.
 12. The electronic device of claim 1, wherein the at least oneprocessor is configured to control the electronic device to: receive thesecond information associated with the image identified based on aresult of analysis of a first text previously transmitted or receivedbetween the first user and the second user, the emotional state of thefirst user, and the emotional state of the second user, as at least partof receiving the information associated with the image identified basedon the first user's emotional state and the second user's emotionalstate.
 13. The electronic device of claim 1, wherein the at least oneprocessor is configured to control the electronic device to: receive,through the input device, an instruction to transmit the displayed imageand the text to an external electronic device being used by the seconduser and transmit the displayed image and the text through thecommunication circuit to the external electronic device.
 14. A method ofoperating an electronic device, the method comprising: receiving a text;transmitting first information about the text to a server; receivingsecond information associated with an image identified based on anemotional state of a first user, the emotional state of the first userbeing identified based on a result of analysis of the text and anemotional state of a second user conversing with the first user; anddisplaying the image based on the second information associated with theimage.
 15. The method of claim 14, wherein receiving the informationassociated with the image includes: receiving third informationassociated with a first sub object corresponding to the emotional stateof the first user and fourth information associated with a second subobject corresponding to the emotional state of the second user, anddisplaying the image based on the second information associated with theimage includes displaying the image including the first sub object andthe second sub object.
 16. The method of claim 15, wherein the first subobject corresponds to a first character designated to be associated withthe first user, and the second sub object corresponds to a secondcharacter designated to be associated with the second user.
 17. Themethod of claim 15, wherein the second information associated with theimage includes first identification information identifying the firstsub object and second identification information identifying the secondsub object, and wherein displaying the image based on the secondinformation associated with the image includes, identifying the firstsub object corresponding to the first identification information fromamong a plurality of stored images, and identifying the second subobject corresponding to the second identification information from amongthe plurality of images.
 18. The method of claim 15, wherein the secondinformation associated with the image includes the emotional state ofthe first user and the emotional state of the second user, and whereindisplaying the image based on the second information associated with theimage includes: identifying the first sub object corresponding to theemotional state of the first user from among a plurality of imagesstored; and identifying the second sub object corresponding to theemotional state of the second user from among the plurality of images.19. The method of claim 18, wherein the emotional state of the firstuser includes a type of emotion of the first user and a degree ofemotion of the first user, and the emotional state of the second userincludes a type of emotion of the second user and a degree of emotion ofthe second user, and wherein displaying the image based on the secondinformation associated with the image includes: identifying the firstsub object corresponding to the type of emotion of the first user andthe degree of emotion of the first user from among a plurality of imagescorresponding to a first character designated to be associated with thefirst user and the second sub object corresponding to the type ofemotion of the second user and the degree of emotion of the second userfrom among a plurality of images corresponding to a second characterdesignated to be associated with the second user.
 20. The method ofclaim 14, wherein the second information associated with the imageincludes fifth information about a position of the first sub object andsixth information about a position of the second sub object, and whereindisplaying the image based on the second information associated with theimage includes placing the first sub object in a first position in theimage based on the fifth information and the second sub object in asecond position in the image based on the sixth information.